2026/07/16 03:01:51 Test passed > Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/16/26 03:00:15.221 Create an experiment with name E2EExperiment-1784170815221955193 < Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/16/26 03:00:15.41 (188ms) > Enter [BeforeEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/16/26 03:00:15.41 ################### Setup before each Pipeline Upload test ##################### ################### Global Setup before each test ##################### Test Context: 0x1fc58a69a200 test-secret-1 created test-secret-2 created test-secret-3 created < Exit [BeforeEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/16/26 03:00:15.48 (70ms) > Enter [It] Upload critical/pipeline_with_artifact_upload_download.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/16/26 03:00:15.48 Uploading pipeline file critical/pipeline_with_artifact_upload_download.yaml Creating temp pipeline file with overridden SDK Version Uploading pipeline with name=e2e-test-1784170815410644270, from file /dspa/test_data/sdk_compiled_pipelines/valid/critical/pipeline_with_artifact_upload_download.yaml Upload of pipeline file 'critical/pipeline_with_artifact_upload_download.yaml' successful Listing pipeline versions for pipeline 33d5b0ef-efef-48e4-9443-756d62489eaf Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/critical/pipeline_with_artifact_upload_download.yaml Returning pipeline run time inputs map[] Create run for pipeline with id: '33d5b0ef-efef-48e4-9443-756d62489eaf' and name: 'e2e-test-1784170815410644270' Create a pipeline run for pipeline with id=33d5b0ef-efef-48e4-9443-756d62489eaf and versionId=15d82a32-9bb0-4402-a501-5042335b0d90 Create a pipeline run body Created Pipeline Run successfully with runId=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Created Pipeline Run with id: e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 for pipeline with id: 33d5b0ef-efef-48e4-9443-756d62489eaf Waiting for pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 to be in one of '[SUCCEEDED SKIPPED FAILED CANCELED]' Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in PENDING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 is in RUNNING state, waiting... Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 reached expected state SUCCEEDED Deserializing expected compiled workflow file 'critical/pipeline_with_artifact_upload_download.yaml' for the pipeline Unmarshalling Expected Workflow YAML Unmarshalled Expected Workflow YAML Fetching updated pipeline run details for run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Get a pipeline run with id=e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Updated pipeline run details Pipeline run succeeded, checking if the number of tasks are what is expected < Exit [It] Upload critical/pipeline_with_artifact_upload_download.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/16/26 03:01:50.881 (1m35.401s) > Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/16/26 03:01:50.882 ################### Global Cleanup after each test ##################### Deleting 1 run(s) Terminate run e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Failed to terminate run e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Terminate run e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Deleting run e21dfc8e-d1e9-4318-9b8a-aa3b9ea5cf95 Deleting 0 experiment(s) Deleting 1 pipeline(s) Deleting pipeline with id=33d5b0ef-efef-48e4-9443-756d62489eaf (cascade=true) Pipeline with id=33d5b0ef-efef-48e4-9443-756d62489eaf, DELETED < Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/16/26 03:01:51.025 (144ms) > Enter [ReportAfterEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/16/26 03:01:51.026 < Exit [ReportAfterEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/16/26 03:01:51.026 (0s) > Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/16/26 03:01:51.026 < Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/16/26 03:01:51.026 (0s) 2026/07/16 03:01:01 Test passed > Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/16/26 03:00:15.222 Create an experiment with name E2EExperiment-1784170815222064336 < Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/16/26 03:00:15.48 (258ms) > Enter [BeforeEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/16/26 03:00:15.48 ################### Setup before each Pipeline Upload test ##################### ################### Global Setup before each test ##################### Test Context: 0xb2271f42500 Looks like test-secret-1 already exists, because creation failed due to secrets "test-secret-1" already exists Looks like test-secret-2 already exists, because creation failed due to secrets "test-secret-2" already exists Looks like test-secret-3 already exists, because creation failed due to secrets "test-secret-3" already exists < Exit [BeforeEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/16/26 03:00:15.606 (125ms) > Enter [It] Upload essential/component_with_pip_index_urls.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/16/26 03:00:15.606 Uploading pipeline file essential/component_with_pip_index_urls.yaml Creating temp pipeline file with overridden SDK Version Uploading pipeline with name=e2e-test-1784170815480821572, from file /dspa/test_data/sdk_compiled_pipelines/valid/essential/component_with_pip_index_urls.yaml Upload of pipeline file 'essential/component_with_pip_index_urls.yaml' successful Listing pipeline versions for pipeline 69d21858-fd8f-44ea-81f7-59e1f7be1136 Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/essential/component_with_pip_index_urls.yaml Returning pipeline run time inputs map[] Create run for pipeline with id: '69d21858-fd8f-44ea-81f7-59e1f7be1136' and name: 'e2e-test-1784170815480821572' Create a pipeline run for pipeline with id=69d21858-fd8f-44ea-81f7-59e1f7be1136 and versionId=94d094d6-619c-4daa-ada4-42f8e1f67d1e Create a pipeline run body Created Pipeline Run successfully with runId=7fd4478a-c881-4966-8e85-d41784ffdf58 Created Pipeline Run with id: 7fd4478a-c881-4966-8e85-d41784ffdf58 for pipeline with id: 69d21858-fd8f-44ea-81f7-59e1f7be1136 Waiting for pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 to be in one of '[SUCCEEDED SKIPPED FAILED CANCELED]' Get a pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 Pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 is in PENDING state, waiting... Get a pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 Pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 is in RUNNING state, waiting... Get a pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 Pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 is in RUNNING state, waiting... Get a pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 Pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 is in RUNNING state, waiting... Get a pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 Pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 is in RUNNING state, waiting... Get a pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 Pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 is in RUNNING state, waiting... Get a pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 Pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 is in RUNNING state, waiting... Get a pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 Pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 is in RUNNING state, waiting... Get a pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 Pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 is in RUNNING state, waiting... Get a pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 Pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 reached expected state SUCCEEDED Deserializing expected compiled workflow file 'essential/component_with_pip_index_urls.yaml' for the pipeline Unmarshalling Expected Workflow YAML Unmarshalled Expected Workflow YAML Fetching updated pipeline run details for run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 Get a pipeline run with id=7fd4478a-c881-4966-8e85-d41784ffdf58 Updated pipeline run details Pipeline run succeeded, checking if the number of tasks are what is expected < Exit [It] Upload essential/component_with_pip_index_urls.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/16/26 03:01:00.96 (45.354s) > Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/16/26 03:01:00.96 ################### Global Cleanup after each test ##################### Deleting 1 run(s) Terminate run 7fd4478a-c881-4966-8e85-d41784ffdf58 Failed to terminate run 7fd4478a-c881-4966-8e85-d41784ffdf58 Terminate run 7fd4478a-c881-4966-8e85-d41784ffdf58 Deleting run 7fd4478a-c881-4966-8e85-d41784ffdf58 Deleting 0 experiment(s) Deleting 1 pipeline(s) Deleting pipeline with id=69d21858-fd8f-44ea-81f7-59e1f7be1136 (cascade=true) Pipeline with id=69d21858-fd8f-44ea-81f7-59e1f7be1136, DELETED < Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/16/26 03:01:01.102 (142ms) > Enter [ReportAfterEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/16/26 03:01:01.102 < Exit [ReportAfterEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/16/26 03:01:01.102 (0s) > Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/16/26 03:01:01.102 < Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/16/26 03:01:01.102 (0s) 2026/07/16 03:01:51 Test passed > Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/16/26 03:00:15.221 Create an experiment with name E2EExperiment-1784170815221958253 < Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/16/26 03:00:15.411 (190ms) > Enter [BeforeEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/16/26 03:00:15.412 ################### Setup before each Pipeline Upload test ##################### ################### Global Setup before each test ##################### Test Context: 0x388d1828700 Looks like test-secret-1 already exists, because creation failed due to secrets "test-secret-1" already exists Looks like test-secret-2 already exists, because creation failed due to secrets "test-secret-2" already exists Looks like test-secret-3 already exists, because creation failed due to secrets "test-secret-3" already exists < Exit [BeforeEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/16/26 03:00:15.531 (119ms) > Enter [It] Upload critical/parallel_for_after_dependency.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/16/26 03:00:15.531 Uploading pipeline file critical/parallel_for_after_dependency.yaml Creating temp pipeline file with overridden SDK Version Uploading pipeline with name=e2e-test-1784170815412118369, from file /dspa/test_data/sdk_compiled_pipelines/valid/critical/parallel_for_after_dependency.yaml Upload of pipeline file 'critical/parallel_for_after_dependency.yaml' successful Listing pipeline versions for pipeline c2a94ecc-20f9-4a10-b767-f46418aca33f Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/critical/parallel_for_after_dependency.yaml Returning pipeline run time inputs map[] Create run for pipeline with id: 'c2a94ecc-20f9-4a10-b767-f46418aca33f' and name: 'e2e-test-1784170815412118369' Create a pipeline run for pipeline with id=c2a94ecc-20f9-4a10-b767-f46418aca33f and versionId=8b630266-cc7b-46c5-8a1f-d58ca59fe752 Create a pipeline run body Created Pipeline Run successfully with runId=d5e9cabe-672b-4592-b66a-658d5075ca1e Created Pipeline Run with id: d5e9cabe-672b-4592-b66a-658d5075ca1e for pipeline with id: c2a94ecc-20f9-4a10-b767-f46418aca33f Waiting for pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e to be in one of '[SUCCEEDED SKIPPED FAILED CANCELED]' Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in PENDING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e is in RUNNING state, waiting... Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e reached expected state SUCCEEDED Deserializing expected compiled workflow file 'critical/parallel_for_after_dependency.yaml' for the pipeline Unmarshalling Expected Workflow YAML Unmarshalled Expected Workflow YAML Fetching updated pipeline run details for run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Get a pipeline run with id=d5e9cabe-672b-4592-b66a-658d5075ca1e Updated pipeline run details Pipeline run succeeded, checking if the number of tasks are what is expected < Exit [It] Upload critical/parallel_for_after_dependency.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/16/26 03:01:51.073 (1m35.542s) > Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/16/26 03:01:51.074 ################### Global Cleanup after each test ##################### Deleting 1 run(s) Terminate run d5e9cabe-672b-4592-b66a-658d5075ca1e Failed to terminate run d5e9cabe-672b-4592-b66a-658d5075ca1e Terminate run d5e9cabe-672b-4592-b66a-658d5075ca1e Deleting run d5e9cabe-672b-4592-b66a-658d5075ca1e Deleting 0 experiment(s) Deleting 1 pipeline(s) Deleting pipeline with id=c2a94ecc-20f9-4a10-b767-f46418aca33f (cascade=true) Pipeline with id=c2a94ecc-20f9-4a10-b767-f46418aca33f, DELETED < Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/16/26 03:01:51.402 (328ms) > Enter [ReportAfterEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/16/26 03:01:51.402 < Exit [ReportAfterEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/16/26 03:01:51.402 (0s) > Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/16/26 03:01:51.402 < Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/16/26 03:01:51.402 (0s) 2026/07/16 03:02:21 Test passed > Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/16/26 03:00:15.221 Create an experiment with name E2EExperiment-1784170815221761278 < Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/16/26 03:00:15.48 (259ms) > Enter [BeforeEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/16/26 03:00:15.481 ################### Setup before each Pipeline Upload test ##################### ################### Global Setup before each test ##################### Test Context: 0x29878796c000 Looks like test-secret-1 already exists, because creation failed due to secrets "test-secret-1" already exists Looks like test-secret-2 already exists, because creation failed due to secrets "test-secret-2" already exists Looks like test-secret-3 already exists, because creation failed due to secrets "test-secret-3" already exists < Exit [BeforeEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/16/26 03:00:15.608 (127ms) > Enter [It] Upload critical/flip_coin.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/16/26 03:00:15.608 Uploading pipeline file critical/flip_coin.yaml Creating temp pipeline file with overridden SDK Version Uploading pipeline with name=e2e-test-1784170815481030557, from file /dspa/test_data/sdk_compiled_pipelines/valid/critical/flip_coin.yaml Upload of pipeline file 'critical/flip_coin.yaml' successful Listing pipeline versions for pipeline e67801e3-84ad-4070-a2e9-ecd3976b8ffa Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/critical/flip_coin.yaml Returning pipeline run time inputs map[] Create run for pipeline with id: 'e67801e3-84ad-4070-a2e9-ecd3976b8ffa' and name: 'e2e-test-1784170815481030557' Create a pipeline run for pipeline with id=e67801e3-84ad-4070-a2e9-ecd3976b8ffa and versionId=5ac4cbb3-44b2-4ee0-ab33-3423f6ccb518 Create a pipeline run body Created Pipeline Run successfully with runId=d72a0132-7ed6-480c-ae45-ddbd564eac1e Created Pipeline Run with id: d72a0132-7ed6-480c-ae45-ddbd564eac1e for pipeline with id: e67801e3-84ad-4070-a2e9-ecd3976b8ffa Waiting for pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e to be in one of '[SUCCEEDED SKIPPED FAILED CANCELED]' Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in PENDING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e is in RUNNING state, waiting... Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e reached expected state SUCCEEDED Deserializing expected compiled workflow file 'critical/flip_coin.yaml' for the pipeline Unmarshalling Expected Workflow YAML Unmarshalled Expected Workflow YAML Fetching updated pipeline run details for run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Get a pipeline run with id=d72a0132-7ed6-480c-ae45-ddbd564eac1e Updated pipeline run details Pipeline run succeeded, checking if the number of tasks are what is expected < Exit [It] Upload critical/flip_coin.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/16/26 03:02:21.219 (2m5.611s) > Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/16/26 03:02:21.219 ################### Global Cleanup after each test ##################### Deleting 1 run(s) Terminate run d72a0132-7ed6-480c-ae45-ddbd564eac1e Failed to terminate run d72a0132-7ed6-480c-ae45-ddbd564eac1e Terminate run d72a0132-7ed6-480c-ae45-ddbd564eac1e Deleting run d72a0132-7ed6-480c-ae45-ddbd564eac1e Deleting 0 experiment(s) Deleting 1 pipeline(s) Deleting pipeline with id=e67801e3-84ad-4070-a2e9-ecd3976b8ffa (cascade=true) Pipeline with id=e67801e3-84ad-4070-a2e9-ecd3976b8ffa, DELETED < Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/16/26 03:02:21.432 (213ms) > Enter [ReportAfterEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/16/26 03:02:21.433 < Exit [ReportAfterEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/16/26 03:02:21.433 (0s) > Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/16/26 03:02:21.433 < Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/16/26 03:02:21.433 (0s) 2026/07/16 03:02:51 Test passed > Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/16/26 03:00:15.222 Create an experiment with name E2EExperiment-1784170815222099607 < Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/16/26 03:00:15.485 (264ms) > Enter [BeforeEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/16/26 03:00:15.485 ################### Setup before each Pipeline Upload test ##################### ################### Global Setup before each test ##################### Test Context: 0x3aebea58380 Looks like test-secret-1 already exists, because creation failed due to secrets "test-secret-1" already exists Looks like test-secret-2 already exists, because creation failed due to secrets "test-secret-2" already exists Looks like test-secret-3 already exists, because creation failed due to secrets "test-secret-3" already exists < Exit [BeforeEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/16/26 03:00:15.6 (115ms) > Enter [It] Upload essential/iris_pipeline_compiled.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/16/26 03:00:15.601 Uploading pipeline file essential/iris_pipeline_compiled.yaml Creating temp pipeline file with overridden SDK Version Uploading pipeline with name=e2e-test-1784170815485980309, from file /dspa/test_data/sdk_compiled_pipelines/valid/essential/iris_pipeline_compiled.yaml Upload of pipeline file 'essential/iris_pipeline_compiled.yaml' successful Listing pipeline versions for pipeline 0aeaad78-71de-4c0b-af68-710fa2a49bbe Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/essential/iris_pipeline_compiled.yaml Returning pipeline run time inputs map[] Create run for pipeline with id: '0aeaad78-71de-4c0b-af68-710fa2a49bbe' and name: 'e2e-test-1784170815485980309' Create a pipeline run for pipeline with id=0aeaad78-71de-4c0b-af68-710fa2a49bbe and versionId=867ce318-e893-4c2c-b99e-db8d3f104ff4 Create a pipeline run body Created Pipeline Run successfully with runId=21c33170-d39e-482d-acf0-7a3bdc40f874 Created Pipeline Run with id: 21c33170-d39e-482d-acf0-7a3bdc40f874 for pipeline with id: 0aeaad78-71de-4c0b-af68-710fa2a49bbe Waiting for pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 to be in one of '[SUCCEEDED SKIPPED FAILED CANCELED]' Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in PENDING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 is in RUNNING state, waiting... Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 reached expected state SUCCEEDED Deserializing expected compiled workflow file 'essential/iris_pipeline_compiled.yaml' for the pipeline Unmarshalling Expected Workflow YAML Unmarshalled Expected Workflow YAML Fetching updated pipeline run details for run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Get a pipeline run with id=21c33170-d39e-482d-acf0-7a3bdc40f874 Updated pipeline run details Pipeline run succeeded, checking if the number of tasks are what is expected < Exit [It] Upload essential/iris_pipeline_compiled.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/16/26 03:02:50.972 (2m35.371s) > Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/16/26 03:02:50.972 ################### Global Cleanup after each test ##################### Deleting 1 run(s) Terminate run 21c33170-d39e-482d-acf0-7a3bdc40f874 Failed to terminate run 21c33170-d39e-482d-acf0-7a3bdc40f874 Terminate run 21c33170-d39e-482d-acf0-7a3bdc40f874 Deleting run 21c33170-d39e-482d-acf0-7a3bdc40f874 Deleting 0 experiment(s) Deleting 1 pipeline(s) Deleting pipeline with id=0aeaad78-71de-4c0b-af68-710fa2a49bbe (cascade=true) Pipeline with id=0aeaad78-71de-4c0b-af68-710fa2a49bbe, DELETED < Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/16/26 03:02:51.126 (154ms) > Enter [ReportAfterEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/16/26 03:02:51.127 < Exit [ReportAfterEach] Upload and Verify Pipeline Run > - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/16/26 03:02:51.127 (0s) > Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/16/26 03:02:51.127 < Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/16/26 03:02:51.127 (0s)