<?xml version="1.0" encoding="UTF-8"?>
  <testsuites tests="5" disabled="0" errors="0" failures="0" time="175.903552336">
      <testsuite name="AIPipelinesE2ETests" package="/dspa/backend/test/end2end" tests="5" disabled="0" skipped="0" errors="0" failures="0" time="175.903552336" timestamp="2026-07-20T09:28:22">
          <properties>
              <property name="SuiteSucceeded" value="true"></property>
              <property name="SuiteHasProgrammaticFocus" value="false"></property>
              <property name="SpecialSuiteFailureReason" value=""></property>
              <property name="SuiteLabels" value="[]"></property>
              <property name="SuiteSemVerConstraints" value="[]"></property>
              <property name="RandomSeed" value="1784539519"></property>
              <property name="RandomizeAllSpecs" value="false"></property>
              <property name="LabelFilter" value="Smoke"></property>
              <property name="SemVerFilter" value=""></property>
              <property name="FocusStrings" value=""></property>
              <property name="SkipStrings" value=""></property>
              <property name="FocusFiles" value=""></property>
              <property name="SkipFiles" value=""></property>
              <property name="FailOnPending" value="false"></property>
              <property name="FailOnEmpty" value="false"></property>
              <property name="FailFast" value="false"></property>
              <property name="FlakeAttempts" value="0"></property>
              <property name="DryRun" value="false"></property>
              <property name="ParallelTotal" value="10"></property>
              <property name="OutputInterceptorMode" value=""></property>
          </properties>
          <testcase name="[It] Upload and Verify Pipeline Run &gt; Upload a pipeline file, run it and verify that pipeline run succeeds Smoke &gt; Upload essential/component_with_pip_index_urls.yaml pipeline [FullRegression, smoke]" classname="AIPipelinesE2ETests" status="passed" time="75.801489655">
              <system-out>2026/07/20 09:29:37 Test passed&#xA;</system-out>
              <system-err>&gt; Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/20/26 09:28:22.157&#xA;Create an experiment with name E2EExperiment-1784539702157746751&#xA;&lt; Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/20/26 09:28:22.331 (174ms)&#xA;&gt; Enter [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/20/26 09:28:22.331&#xA;################### Setup before each Pipeline Upload test #####################&#xA;################### Global Setup before each test #####################&#xA;Test Context: 0xfd3b2eb0180&#xA;Looks like test-secret-1 already exists, because creation failed due to secrets &#34;test-secret-1&#34; already exists&#xA;Looks like test-secret-2 already exists, because creation failed due to secrets &#34;test-secret-2&#34; already exists&#xA;Looks like test-secret-3 already exists, because creation failed due to secrets &#34;test-secret-3&#34; already exists&#xA;&lt; Exit [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/20/26 09:28:22.372 (41ms)&#xA;&gt; Enter [It] Upload essential/component_with_pip_index_urls.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/20/26 09:28:22.372&#xA;Uploading pipeline file essential/component_with_pip_index_urls.yaml&#xA;Creating temp pipeline file with overridden SDK Version&#xA;Uploading pipeline with name=e2e-test-1784539702331898655, from file /dspa/test_data/sdk_compiled_pipelines/valid/essential/component_with_pip_index_urls.yaml&#xA;Upload of pipeline file &#39;essential/component_with_pip_index_urls.yaml&#39; successful&#xA;Listing pipeline versions for pipeline edd04668-21d7-4d83-8b33-ca392d2dcfeb&#xA;Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/essential/component_with_pip_index_urls.yaml&#xA;Returning pipeline run time inputs map[]&#xA;Create run for pipeline with id: &#39;edd04668-21d7-4d83-8b33-ca392d2dcfeb&#39; and name: &#39;e2e-test-1784539702331898655&#39;&#xA;Create a pipeline run for pipeline with id=edd04668-21d7-4d83-8b33-ca392d2dcfeb and versionId=4a30849d-f7ee-404e-890e-5f57fad887e7&#xA;Create a pipeline run body&#xA;Created Pipeline Run successfully with runId=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Created Pipeline Run with id: 26897100-edb4-4e25-bde6-d085e5f56e56 for pipeline with id: edd04668-21d7-4d83-8b33-ca392d2dcfeb&#xA;Waiting for pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 to be in one of &#39;[SUCCEEDED SKIPPED FAILED CANCELED]&#39;&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in PENDING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56 reached expected state SUCCEEDED&#xA;Deserializing expected compiled workflow file &#39;essential/component_with_pip_index_urls.yaml&#39; for the pipeline&#xA;Unmarshalling Expected Workflow YAML&#xA;Unmarshalled Expected Workflow YAML&#xA;Fetching updated pipeline run details for run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Get a pipeline run with id=26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Updated pipeline run details&#xA;Pipeline run succeeded, checking if the number of tasks are what is expected&#xA;&lt; Exit [It] Upload essential/component_with_pip_index_urls.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/20/26 09:29:37.754 (1m15.382s)&#xA;&gt; Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/20/26 09:29:37.754&#xA;################### Global Cleanup after each test #####################&#xA;Deleting 1 run(s)&#xA;Terminate run 26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Failed to terminate run 26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Terminate run 26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Deleting run 26897100-edb4-4e25-bde6-d085e5f56e56&#xA;Deleting 0 experiment(s)&#xA;Deleting 1 pipeline(s)&#xA;Deleting pipeline with id=edd04668-21d7-4d83-8b33-ca392d2dcfeb (cascade=true)&#xA;Pipeline with id=edd04668-21d7-4d83-8b33-ca392d2dcfeb, DELETED&#xA;&lt; Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/20/26 09:29:37.959 (204ms)&#xA;&gt; Enter [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/20/26 09:29:37.959&#xA;&lt; Exit [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/20/26 09:29:37.959 (0s)&#xA;&gt; Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/20/26 09:29:37.959&#xA;&lt; Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/20/26 09:29:37.959 (0s)&#xA;</system-err>
          </testcase>
          <testcase name="[It] Upload and Verify Pipeline Run &gt; Upload a pipeline file, run it and verify that pipeline run succeeds Smoke &gt; Upload critical/parallel_for_after_dependency.yaml pipeline [FullRegression, smoke]" classname="AIPipelinesE2ETests" status="passed" time="115.775315845">
              <system-out>2026/07/20 09:30:17 Test passed&#xA;</system-out>
              <system-err>&gt; Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/20/26 09:28:22.157&#xA;Create an experiment with name E2EExperiment-1784539702157725170&#xA;&lt; Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/20/26 09:28:22.326 (169ms)&#xA;&gt; Enter [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/20/26 09:28:22.326&#xA;################### Setup before each Pipeline Upload test #####################&#xA;################### Global Setup before each test #####################&#xA;Test Context: 0xbbc6058800&#xA;Looks like test-secret-1 already exists, because creation failed due to secrets &#34;test-secret-1&#34; already exists&#xA;Looks like test-secret-2 already exists, because creation failed due to secrets &#34;test-secret-2&#34; already exists&#xA;Looks like test-secret-3 already exists, because creation failed due to secrets &#34;test-secret-3&#34; already exists&#xA;&lt; Exit [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/20/26 09:28:22.37 (44ms)&#xA;&gt; Enter [It] Upload critical/parallel_for_after_dependency.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/20/26 09:28:22.37&#xA;Uploading pipeline file critical/parallel_for_after_dependency.yaml&#xA;Creating temp pipeline file with overridden SDK Version&#xA;Uploading pipeline with name=e2e-test-1784539702326492452, from file /dspa/test_data/sdk_compiled_pipelines/valid/critical/parallel_for_after_dependency.yaml&#xA;Upload of pipeline file &#39;critical/parallel_for_after_dependency.yaml&#39; successful&#xA;Listing pipeline versions for pipeline 72ecd1e1-57da-4274-985b-bb4fa12293d2&#xA;Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/critical/parallel_for_after_dependency.yaml&#xA;Returning pipeline run time inputs map[]&#xA;Create run for pipeline with id: &#39;72ecd1e1-57da-4274-985b-bb4fa12293d2&#39; and name: &#39;e2e-test-1784539702326492452&#39;&#xA;Create a pipeline run for pipeline with id=72ecd1e1-57da-4274-985b-bb4fa12293d2 and versionId=f7073c01-a2c5-4a9f-870f-65f75f23688c&#xA;Create a pipeline run body&#xA;Created Pipeline Run successfully with runId=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Created Pipeline Run with id: cac68e5f-2a85-4ac9-82e6-5ff90a4836fc for pipeline with id: 72ecd1e1-57da-4274-985b-bb4fa12293d2&#xA;Waiting for pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc to be in one of &#39;[SUCCEEDED SKIPPED FAILED CANCELED]&#39;&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in PENDING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc reached expected state SUCCEEDED&#xA;Deserializing expected compiled workflow file &#39;critical/parallel_for_after_dependency.yaml&#39; for the pipeline&#xA;Unmarshalling Expected Workflow YAML&#xA;Unmarshalled Expected Workflow YAML&#xA;Fetching updated pipeline run details for run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Get a pipeline run with id=cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Updated pipeline run details&#xA;Pipeline run succeeded, checking if the number of tasks are what is expected&#xA;&lt; Exit [It] Upload critical/parallel_for_after_dependency.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/20/26 09:30:17.805 (1m55.435s)&#xA;&gt; Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/20/26 09:30:17.805&#xA;################### Global Cleanup after each test #####################&#xA;Deleting 1 run(s)&#xA;Terminate run cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Failed to terminate run cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Terminate run cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Deleting run cac68e5f-2a85-4ac9-82e6-5ff90a4836fc&#xA;Deleting 0 experiment(s)&#xA;Deleting 1 pipeline(s)&#xA;Deleting pipeline with id=72ecd1e1-57da-4274-985b-bb4fa12293d2 (cascade=true)&#xA;Pipeline with id=72ecd1e1-57da-4274-985b-bb4fa12293d2, DELETED&#xA;&lt; Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/20/26 09:30:17.932 (127ms)&#xA;&gt; Enter [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/20/26 09:30:17.933&#xA;&lt; Exit [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/20/26 09:30:17.933 (0s)&#xA;&gt; Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/20/26 09:30:17.933&#xA;&lt; Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/20/26 09:30:17.933 (0s)&#xA;</system-err>
          </testcase>
          <testcase name="[It] Upload and Verify Pipeline Run &gt; Upload a pipeline file, run it and verify that pipeline run succeeds Smoke &gt; Upload critical/pipeline_with_artifact_upload_download.yaml pipeline [FullRegression, smoke]" classname="AIPipelinesE2ETests" status="passed" time="120.786553377">
              <system-out>2026/07/20 09:30:22 Test passed&#xA;</system-out>
              <system-err>&gt; Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/20/26 09:28:22.157&#xA;Create an experiment with name E2EExperiment-1784539702157651406&#xA;&lt; Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/20/26 09:28:22.326 (169ms)&#xA;&gt; Enter [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/20/26 09:28:22.326&#xA;################### Setup before each Pipeline Upload test #####################&#xA;################### Global Setup before each test #####################&#xA;Test Context: 0x111611f4f080&#xA;test-secret-1 created&#xA;test-secret-2 created&#xA;test-secret-3 created&#xA;&lt; Exit [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/20/26 09:28:22.356 (31ms)&#xA;&gt; Enter [It] Upload critical/pipeline_with_artifact_upload_download.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/20/26 09:28:22.357&#xA;Uploading pipeline file critical/pipeline_with_artifact_upload_download.yaml&#xA;Creating temp pipeline file with overridden SDK Version&#xA;Uploading pipeline with name=e2e-test-1784539702326474651, from file /dspa/test_data/sdk_compiled_pipelines/valid/critical/pipeline_with_artifact_upload_download.yaml&#xA;Upload of pipeline file &#39;critical/pipeline_with_artifact_upload_download.yaml&#39; successful&#xA;Listing pipeline versions for pipeline 069aca8e-793e-4353-ba0d-a20108feb88a&#xA;Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/critical/pipeline_with_artifact_upload_download.yaml&#xA;Returning pipeline run time inputs map[]&#xA;Create run for pipeline with id: &#39;069aca8e-793e-4353-ba0d-a20108feb88a&#39; and name: &#39;e2e-test-1784539702326474651&#39;&#xA;Create a pipeline run for pipeline with id=069aca8e-793e-4353-ba0d-a20108feb88a and versionId=e6413e41-baac-4a7f-81c9-057b9afc533c&#xA;Create a pipeline run body&#xA;Created Pipeline Run successfully with runId=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Created Pipeline Run with id: 66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 for pipeline with id: 069aca8e-793e-4353-ba0d-a20108feb88a&#xA;Waiting for pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 to be in one of &#39;[SUCCEEDED SKIPPED FAILED CANCELED]&#39;&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in PENDING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748 reached expected state SUCCEEDED&#xA;Deserializing expected compiled workflow file &#39;critical/pipeline_with_artifact_upload_download.yaml&#39; for the pipeline&#xA;Unmarshalling Expected Workflow YAML&#xA;Unmarshalled Expected Workflow YAML&#xA;Fetching updated pipeline run details for run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Get a pipeline run with id=66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Updated pipeline run details&#xA;Pipeline run succeeded, checking if the number of tasks are what is expected&#xA;&lt; Exit [It] Upload critical/pipeline_with_artifact_upload_download.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/20/26 09:30:22.822 (2m0.465s)&#xA;&gt; Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/20/26 09:30:22.822&#xA;################### Global Cleanup after each test #####################&#xA;Deleting 1 run(s)&#xA;Terminate run 66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Failed to terminate run 66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Terminate run 66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Deleting run 66fa22c7-c0cc-47f0-80d1-1d8d4e44f748&#xA;Deleting 0 experiment(s)&#xA;Deleting 1 pipeline(s)&#xA;Deleting pipeline with id=069aca8e-793e-4353-ba0d-a20108feb88a (cascade=true)&#xA;Pipeline with id=069aca8e-793e-4353-ba0d-a20108feb88a, DELETED&#xA;&lt; Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/20/26 09:30:22.944 (121ms)&#xA;&gt; Enter [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/20/26 09:30:22.944&#xA;&lt; Exit [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/20/26 09:30:22.944 (0s)&#xA;&gt; Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/20/26 09:30:22.944&#xA;&lt; Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/20/26 09:30:22.944 (0s)&#xA;</system-err>
          </testcase>
          <testcase name="[It] Upload and Verify Pipeline Run &gt; Upload a pipeline file, run it and verify that pipeline run succeeds Smoke &gt; Upload critical/flip_coin.yaml pipeline [FullRegression, smoke]" classname="AIPipelinesE2ETests" status="passed" time="150.846553024">
              <system-out>2026/07/20 09:30:53 Test passed&#xA;</system-out>
              <system-err>&gt; Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/20/26 09:28:22.157&#xA;Create an experiment with name E2EExperiment-1784539702157585233&#xA;&lt; Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/20/26 09:28:22.33 (173ms)&#xA;&gt; Enter [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/20/26 09:28:22.331&#xA;################### Setup before each Pipeline Upload test #####################&#xA;################### Global Setup before each test #####################&#xA;Test Context: 0x3ef229aa0880&#xA;Looks like test-secret-1 already exists, because creation failed due to secrets &#34;test-secret-1&#34; already exists&#xA;Looks like test-secret-2 already exists, because creation failed due to secrets &#34;test-secret-2&#34; already exists&#xA;Looks like test-secret-3 already exists, because creation failed due to secrets &#34;test-secret-3&#34; already exists&#xA;&lt; Exit [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/20/26 09:28:22.371 (41ms)&#xA;&gt; Enter [It] Upload critical/flip_coin.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/20/26 09:28:22.371&#xA;Uploading pipeline file critical/flip_coin.yaml&#xA;Creating temp pipeline file with overridden SDK Version&#xA;Uploading pipeline with name=e2e-test-1784539702331163086, from file /dspa/test_data/sdk_compiled_pipelines/valid/critical/flip_coin.yaml&#xA;Upload of pipeline file &#39;critical/flip_coin.yaml&#39; successful&#xA;Listing pipeline versions for pipeline 3871891e-2efb-48d2-8913-80f9762e2729&#xA;Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/critical/flip_coin.yaml&#xA;Returning pipeline run time inputs map[]&#xA;Create run for pipeline with id: &#39;3871891e-2efb-48d2-8913-80f9762e2729&#39; and name: &#39;e2e-test-1784539702331163086&#39;&#xA;Create a pipeline run for pipeline with id=3871891e-2efb-48d2-8913-80f9762e2729 and versionId=c2b4792b-14b1-4f26-b201-ae791e9f6922&#xA;Create a pipeline run body&#xA;Created Pipeline Run successfully with runId=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Created Pipeline Run with id: 0da01d3f-e2fe-44b9-85c8-d66736716992 for pipeline with id: 3871891e-2efb-48d2-8913-80f9762e2729&#xA;Waiting for pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 to be in one of &#39;[SUCCEEDED SKIPPED FAILED CANCELED]&#39;&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in PENDING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992 reached expected state SUCCEEDED&#xA;Deserializing expected compiled workflow file &#39;critical/flip_coin.yaml&#39; for the pipeline&#xA;Unmarshalling Expected Workflow YAML&#xA;Unmarshalled Expected Workflow YAML&#xA;Fetching updated pipeline run details for run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Get a pipeline run with id=0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Updated pipeline run details&#xA;Pipeline run succeeded, checking if the number of tasks are what is expected&#xA;&lt; Exit [It] Upload critical/flip_coin.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/20/26 09:30:52.861 (2m30.49s)&#xA;&gt; Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/20/26 09:30:52.861&#xA;################### Global Cleanup after each test #####################&#xA;Deleting 1 run(s)&#xA;Terminate run 0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Failed to terminate run 0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Terminate run 0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Deleting run 0da01d3f-e2fe-44b9-85c8-d66736716992&#xA;Deleting 0 experiment(s)&#xA;Deleting 1 pipeline(s)&#xA;Deleting pipeline with id=3871891e-2efb-48d2-8913-80f9762e2729 (cascade=true)&#xA;Pipeline with id=3871891e-2efb-48d2-8913-80f9762e2729, DELETED&#xA;&lt; Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/20/26 09:30:53.003 (142ms)&#xA;&gt; Enter [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/20/26 09:30:53.004&#xA;&lt; Exit [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/20/26 09:30:53.004 (0s)&#xA;&gt; Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/20/26 09:30:53.004&#xA;&lt; Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/20/26 09:30:53.004 (0s)&#xA;</system-err>
          </testcase>
          <testcase name="[It] Upload and Verify Pipeline Run &gt; Upload a pipeline file, run it and verify that pipeline run succeeds Smoke &gt; Upload essential/iris_pipeline_compiled.yaml pipeline [FullRegression, smoke]" classname="AIPipelinesE2ETests" status="passed" time="175.778848382">
              <system-out>2026/07/20 09:31:17 Test passed&#xA;</system-out>
              <system-err>&gt; Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/20/26 09:28:22.157&#xA;Create an experiment with name E2EExperiment-1784539702157691368&#xA;&lt; Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/20/26 09:28:22.326 (169ms)&#xA;&gt; Enter [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/20/26 09:28:22.326&#xA;################### Setup before each Pipeline Upload test #####################&#xA;################### Global Setup before each test #####################&#xA;Test Context: 0x2db3fc578980&#xA;Looks like test-secret-1 already exists, because creation failed due to secrets &#34;test-secret-1&#34; already exists&#xA;Looks like test-secret-2 already exists, because creation failed due to secrets &#34;test-secret-2&#34; already exists&#xA;Looks like test-secret-3 already exists, because creation failed due to secrets &#34;test-secret-3&#34; already exists&#xA;&lt; Exit [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/20/26 09:28:22.369 (43ms)&#xA;&gt; Enter [It] Upload essential/iris_pipeline_compiled.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/20/26 09:28:22.369&#xA;Uploading pipeline file essential/iris_pipeline_compiled.yaml&#xA;Creating temp pipeline file with overridden SDK Version&#xA;Uploading pipeline with name=e2e-test-1784539702326459450, from file /dspa/test_data/sdk_compiled_pipelines/valid/essential/iris_pipeline_compiled.yaml&#xA;Upload of pipeline file &#39;essential/iris_pipeline_compiled.yaml&#39; successful&#xA;Listing pipeline versions for pipeline d8aca500-8738-4a95-903f-a208e35d1357&#xA;Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/essential/iris_pipeline_compiled.yaml&#xA;Returning pipeline run time inputs map[]&#xA;Create run for pipeline with id: &#39;d8aca500-8738-4a95-903f-a208e35d1357&#39; and name: &#39;e2e-test-1784539702326459450&#39;&#xA;Create a pipeline run for pipeline with id=d8aca500-8738-4a95-903f-a208e35d1357 and versionId=a5754a54-43f0-413a-8ee4-5df7a867a14b&#xA;Create a pipeline run body&#xA;Created Pipeline Run successfully with runId=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Created Pipeline Run with id: d36e9f89-7b51-4ea9-b423-767afd3e6055 for pipeline with id: d8aca500-8738-4a95-903f-a208e35d1357&#xA;Waiting for pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 to be in one of &#39;[SUCCEEDED SKIPPED FAILED CANCELED]&#39;&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in PENDING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055 reached expected state SUCCEEDED&#xA;Deserializing expected compiled workflow file &#39;essential/iris_pipeline_compiled.yaml&#39; for the pipeline&#xA;Unmarshalling Expected Workflow YAML&#xA;Unmarshalled Expected Workflow YAML&#xA;Fetching updated pipeline run details for run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Get a pipeline run with id=d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Updated pipeline run details&#xA;Pipeline run succeeded, checking if the number of tasks are what is expected&#xA;&lt; Exit [It] Upload essential/iris_pipeline_compiled.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/20/26 09:31:17.802 (2m55.434s)&#xA;&gt; Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/20/26 09:31:17.803&#xA;################### Global Cleanup after each test #####################&#xA;Deleting 1 run(s)&#xA;Terminate run d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Failed to terminate run d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Terminate run d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Deleting run d36e9f89-7b51-4ea9-b423-767afd3e6055&#xA;Deleting 0 experiment(s)&#xA;Deleting 1 pipeline(s)&#xA;Deleting pipeline with id=d8aca500-8738-4a95-903f-a208e35d1357 (cascade=true)&#xA;Pipeline with id=d8aca500-8738-4a95-903f-a208e35d1357, DELETED&#xA;&lt; Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/20/26 09:31:17.936 (133ms)&#xA;&gt; Enter [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/20/26 09:31:17.936&#xA;&lt; Exit [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/20/26 09:31:17.936 (0s)&#xA;&gt; Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/20/26 09:31:17.937&#xA;&lt; Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/20/26 09:31:17.937 (0s)&#xA;</system-err>
          </testcase>
      </testsuite>
  </testsuites>