<?xml version="1.0" encoding="UTF-8"?>
  <testsuites tests="5" disabled="0" errors="0" failures="0" time="195.973636507">
      <testsuite name="AIPipelinesE2ETests" package="/dspa/backend/test/end2end" tests="5" disabled="0" skipped="0" errors="0" failures="0" time="195.973636507" timestamp="2026-07-01T17:35:26">
          <properties>
              <property name="SuiteSucceeded" value="true"></property>
              <property name="SuiteHasProgrammaticFocus" value="false"></property>
              <property name="SpecialSuiteFailureReason" value=""></property>
              <property name="SuiteLabels" value="[]"></property>
              <property name="SuiteSemVerConstraints" value="[]"></property>
              <property name="RandomSeed" value="1782927165"></property>
              <property name="RandomizeAllSpecs" value="false"></property>
              <property name="LabelFilter" value="Smoke"></property>
              <property name="SemVerFilter" value=""></property>
              <property name="FocusStrings" value=""></property>
              <property name="SkipStrings" value=""></property>
              <property name="FocusFiles" value=""></property>
              <property name="SkipFiles" value=""></property>
              <property name="FailOnPending" value="false"></property>
              <property name="FailOnEmpty" value="false"></property>
              <property name="FailFast" value="false"></property>
              <property name="FlakeAttempts" value="0"></property>
              <property name="DryRun" value="false"></property>
              <property name="ParallelTotal" value="10"></property>
              <property name="OutputInterceptorMode" value=""></property>
          </properties>
          <testcase name="[It] Upload and Verify Pipeline Run &gt; Upload a pipeline file, run it and verify that pipeline run succeeds Smoke &gt; Upload critical/flip_coin.yaml pipeline [FullRegression, smoke]" classname="AIPipelinesE2ETests" status="passed" time="165.976755248">
              <system-out>2026/07/01 17:38:12 Test passed&#xA;</system-out>
              <system-err>&gt; Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/01/26 17:35:26.176&#xA;Create an experiment with name E2EExperiment-1782927326176569124&#xA;&lt; Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/01/26 17:35:26.46 (284ms)&#xA;&gt; Enter [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/01/26 17:35:26.46&#xA;################### Setup before each Pipeline Upload test #####################&#xA;################### Global Setup before each test #####################&#xA;Test Context: 0x255ddc94e900&#xA;Looks like test-secret-1 already exists, because creation failed due to secrets &#34;test-secret-1&#34; already exists&#xA;Looks like test-secret-2 already exists, because creation failed due to secrets &#34;test-secret-2&#34; already exists&#xA;Looks like test-secret-3 already exists, because creation failed due to secrets &#34;test-secret-3&#34; already exists&#xA;&lt; Exit [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/01/26 17:35:26.52 (59ms)&#xA;&gt; Enter [It] Upload critical/flip_coin.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/01/26 17:35:26.52&#xA;Uploading pipeline file critical/flip_coin.yaml&#xA;Creating temp pipeline file with overridden SDK Version&#xA;Uploading pipeline with name=e2e-test-1782927326460791708, from file /dspa/test_data/sdk_compiled_pipelines/valid/critical/flip_coin.yaml&#xA;Upload of pipeline file &#39;critical/flip_coin.yaml&#39; successful&#xA;Listing pipeline versions for pipeline 36d3f1c4-1bc0-42ac-8f0d-2416bbe5c61d&#xA;Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/critical/flip_coin.yaml&#xA;Returning pipeline run time inputs map[]&#xA;Create run for pipeline with id: &#39;36d3f1c4-1bc0-42ac-8f0d-2416bbe5c61d&#39; and name: &#39;e2e-test-1782927326460791708&#39;&#xA;Create a pipeline run for pipeline with id=36d3f1c4-1bc0-42ac-8f0d-2416bbe5c61d and versionId=237df0c1-e3ef-4861-a636-5c531f592bc3&#xA;Create a pipeline run body&#xA;Created Pipeline Run successfully with runId=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Created Pipeline Run with id: cb29ee5f-675a-4377-86ff-b99f1e024365 for pipeline with id: 36d3f1c4-1bc0-42ac-8f0d-2416bbe5c61d&#xA;Waiting for pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 to be in one of &#39;[SUCCEEDED SKIPPED FAILED CANCELED]&#39;&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in PENDING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365 reached expected state SUCCEEDED&#xA;Deserializing expected compiled workflow file &#39;critical/flip_coin.yaml&#39; for the pipeline&#xA;Unmarshalling Expected Workflow YAML&#xA;Unmarshalled Expected Workflow YAML&#xA;Fetching updated pipeline run details for run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Get a pipeline run with id=cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Updated pipeline run details&#xA;Pipeline run succeeded, checking if the number of tasks are what is expected&#xA;&lt; Exit [It] Upload critical/flip_coin.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/01/26 17:38:12.026 (2m45.506s)&#xA;&gt; Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/01/26 17:38:12.027&#xA;################### Global Cleanup after each test #####################&#xA;Deleting 1 run(s)&#xA;Terminate run cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Failed to terminate run cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Terminate run cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Deleting run cb29ee5f-675a-4377-86ff-b99f1e024365&#xA;Deleting 0 experiment(s)&#xA;Deleting 1 pipeline(s)&#xA;Deleting pipeline with id=36d3f1c4-1bc0-42ac-8f0d-2416bbe5c61d (cascade=true)&#xA;Pipeline with id=36d3f1c4-1bc0-42ac-8f0d-2416bbe5c61d, DELETED&#xA;&lt; Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/01/26 17:38:12.153 (126ms)&#xA;&gt; Enter [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/01/26 17:38:12.153&#xA;&lt; Exit [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/01/26 17:38:12.153 (0s)&#xA;&gt; Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/01/26 17:38:12.153&#xA;&lt; Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/01/26 17:38:12.153 (0s)&#xA;</system-err>
          </testcase>
          <testcase name="[It] Upload and Verify Pipeline Run &gt; Upload a pipeline file, run it and verify that pipeline run succeeds Smoke &gt; Upload essential/component_with_pip_index_urls.yaml pipeline [FullRegression, smoke]" classname="AIPipelinesE2ETests" status="passed" time="90.872861609">
              <system-out>2026/07/01 17:36:57 Test passed&#xA;</system-out>
              <system-err>&gt; Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/01/26 17:35:26.178&#xA;Create an experiment with name E2EExperiment-1782927326178610109&#xA;&lt; Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/01/26 17:35:26.462 (284ms)&#xA;&gt; Enter [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/01/26 17:35:26.462&#xA;################### Setup before each Pipeline Upload test #####################&#xA;################### Global Setup before each test #####################&#xA;Test Context: 0x17c4a40a2a80&#xA;Looks like test-secret-1 already exists, because creation failed due to secrets &#34;test-secret-1&#34; already exists&#xA;Looks like test-secret-2 already exists, because creation failed due to secrets &#34;test-secret-2&#34; already exists&#xA;Looks like test-secret-3 already exists, because creation failed due to secrets &#34;test-secret-3&#34; already exists&#xA;&lt; Exit [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/01/26 17:35:26.523 (61ms)&#xA;&gt; Enter [It] Upload essential/component_with_pip_index_urls.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/01/26 17:35:26.524&#xA;Uploading pipeline file essential/component_with_pip_index_urls.yaml&#xA;Creating temp pipeline file with overridden SDK Version&#xA;Uploading pipeline with name=e2e-test-1782927326462627451, from file /dspa/test_data/sdk_compiled_pipelines/valid/essential/component_with_pip_index_urls.yaml&#xA;Upload of pipeline file &#39;essential/component_with_pip_index_urls.yaml&#39; successful&#xA;Listing pipeline versions for pipeline 5bfe5d38-9f6b-421e-86be-cc97307602a9&#xA;Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/essential/component_with_pip_index_urls.yaml&#xA;Returning pipeline run time inputs map[]&#xA;Create run for pipeline with id: &#39;5bfe5d38-9f6b-421e-86be-cc97307602a9&#39; and name: &#39;e2e-test-1782927326462627451&#39;&#xA;Create a pipeline run for pipeline with id=5bfe5d38-9f6b-421e-86be-cc97307602a9 and versionId=684f9ef8-1e07-4f9f-ab85-6fa2cfd1f510&#xA;Create a pipeline run body&#xA;Created Pipeline Run successfully with runId=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Created Pipeline Run with id: f64cce8a-f9ff-468a-a238-f55899151eb1 for pipeline with id: 5bfe5d38-9f6b-421e-86be-cc97307602a9&#xA;Waiting for pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 to be in one of &#39;[SUCCEEDED SKIPPED FAILED CANCELED]&#39;&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in PENDING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1 reached expected state SUCCEEDED&#xA;Deserializing expected compiled workflow file &#39;essential/component_with_pip_index_urls.yaml&#39; for the pipeline&#xA;Unmarshalling Expected Workflow YAML&#xA;Unmarshalled Expected Workflow YAML&#xA;Fetching updated pipeline run details for run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Get a pipeline run with id=f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Updated pipeline run details&#xA;Pipeline run succeeded, checking if the number of tasks are what is expected&#xA;&lt; Exit [It] Upload essential/component_with_pip_index_urls.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/01/26 17:36:56.908 (1m30.385s)&#xA;&gt; Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/01/26 17:36:56.908&#xA;################### Global Cleanup after each test #####################&#xA;Deleting 1 run(s)&#xA;Terminate run f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Failed to terminate run f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Terminate run f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Deleting run f64cce8a-f9ff-468a-a238-f55899151eb1&#xA;Deleting 0 experiment(s)&#xA;Deleting 1 pipeline(s)&#xA;Deleting pipeline with id=5bfe5d38-9f6b-421e-86be-cc97307602a9 (cascade=true)&#xA;Pipeline with id=5bfe5d38-9f6b-421e-86be-cc97307602a9, DELETED&#xA;&lt; Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/01/26 17:36:57.051 (142ms)&#xA;&gt; Enter [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/01/26 17:36:57.051&#xA;&lt; Exit [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/01/26 17:36:57.051 (0s)&#xA;&gt; Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/01/26 17:36:57.051&#xA;&lt; Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/01/26 17:36:57.052 (0s)&#xA;</system-err>
          </testcase>
          <testcase name="[It] Upload and Verify Pipeline Run &gt; Upload a pipeline file, run it and verify that pipeline run succeeds Smoke &gt; Upload critical/parallel_for_after_dependency.yaml pipeline [FullRegression, smoke]" classname="AIPipelinesE2ETests" status="passed" time="120.957383218">
              <system-out>2026/07/01 17:37:27 Test passed&#xA;</system-out>
              <system-err>&gt; Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/01/26 17:35:26.176&#xA;Create an experiment with name E2EExperiment-1782927326176724533&#xA;&lt; Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/01/26 17:35:26.46 (284ms)&#xA;&gt; Enter [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/01/26 17:35:26.46&#xA;################### Setup before each Pipeline Upload test #####################&#xA;################### Global Setup before each test #####################&#xA;Test Context: 0x1482a598a180&#xA;Looks like test-secret-1 already exists, because creation failed due to secrets &#34;test-secret-1&#34; already exists&#xA;Looks like test-secret-2 already exists, because creation failed due to secrets &#34;test-secret-2&#34; already exists&#xA;Looks like test-secret-3 already exists, because creation failed due to secrets &#34;test-secret-3&#34; already exists&#xA;&lt; Exit [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/01/26 17:35:26.525 (64ms)&#xA;&gt; Enter [It] Upload critical/parallel_for_after_dependency.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/01/26 17:35:26.525&#xA;Uploading pipeline file critical/parallel_for_after_dependency.yaml&#xA;Creating temp pipeline file with overridden SDK Version&#xA;Uploading pipeline with name=e2e-test-1782927326461019682, from file /dspa/test_data/sdk_compiled_pipelines/valid/critical/parallel_for_after_dependency.yaml&#xA;Upload of pipeline file &#39;critical/parallel_for_after_dependency.yaml&#39; successful&#xA;Listing pipeline versions for pipeline 1d72c956-f6c0-4891-a628-2f89f1a1693e&#xA;Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/critical/parallel_for_after_dependency.yaml&#xA;Returning pipeline run time inputs map[]&#xA;Create run for pipeline with id: &#39;1d72c956-f6c0-4891-a628-2f89f1a1693e&#39; and name: &#39;e2e-test-1782927326461019682&#39;&#xA;Create a pipeline run for pipeline with id=1d72c956-f6c0-4891-a628-2f89f1a1693e and versionId=c48f9f2d-30f5-4ac4-9d20-6954cbe12891&#xA;Create a pipeline run body&#xA;Created Pipeline Run successfully with runId=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Created Pipeline Run with id: 87b7a10b-c66c-475c-bf2f-1755ef1721af for pipeline with id: 1d72c956-f6c0-4891-a628-2f89f1a1693e&#xA;Waiting for pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af to be in one of &#39;[SUCCEEDED SKIPPED FAILED CANCELED]&#39;&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in PENDING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af is in RUNNING state, waiting...&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af reached expected state SUCCEEDED&#xA;Deserializing expected compiled workflow file &#39;critical/parallel_for_after_dependency.yaml&#39; for the pipeline&#xA;Unmarshalling Expected Workflow YAML&#xA;Unmarshalled Expected Workflow YAML&#xA;Fetching updated pipeline run details for run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Get a pipeline run with id=87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Updated pipeline run details&#xA;Pipeline run succeeded, checking if the number of tasks are what is expected&#xA;&lt; Exit [It] Upload critical/parallel_for_after_dependency.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/01/26 17:37:26.931 (2m0.407s)&#xA;&gt; Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/01/26 17:37:26.932&#xA;################### Global Cleanup after each test #####################&#xA;Deleting 1 run(s)&#xA;Terminate run 87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Failed to terminate run 87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Terminate run 87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Deleting run 87b7a10b-c66c-475c-bf2f-1755ef1721af&#xA;Deleting 0 experiment(s)&#xA;Deleting 1 pipeline(s)&#xA;Deleting pipeline with id=1d72c956-f6c0-4891-a628-2f89f1a1693e (cascade=true)&#xA;Pipeline with id=1d72c956-f6c0-4891-a628-2f89f1a1693e, DELETED&#xA;&lt; Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/01/26 17:37:27.133 (202ms)&#xA;&gt; Enter [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/01/26 17:37:27.134&#xA;&lt; Exit [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/01/26 17:37:27.134 (0s)&#xA;&gt; Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/01/26 17:37:27.134&#xA;&lt; Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/01/26 17:37:27.134 (0s)&#xA;</system-err>
          </testcase>
          <testcase name="[It] Upload and Verify Pipeline Run &gt; Upload a pipeline file, run it and verify that pipeline run succeeds Smoke &gt; Upload critical/pipeline_with_artifact_upload_download.yaml pipeline [FullRegression, smoke]" classname="AIPipelinesE2ETests" status="passed" time="140.895960978">
              <system-out>2026/07/01 17:37:47 Test passed&#xA;</system-out>
              <system-err>&gt; Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/01/26 17:35:26.177&#xA;Create an experiment with name E2EExperiment-1782927326177027042&#xA;&lt; Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/01/26 17:35:26.463 (287ms)&#xA;&gt; Enter [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/01/26 17:35:26.467&#xA;################### Setup before each Pipeline Upload test #####################&#xA;################### Global Setup before each test #####################&#xA;Test Context: 0x801f2fd0480&#xA;Looks like test-secret-1 already exists, because creation failed due to secrets &#34;test-secret-1&#34; already exists&#xA;Looks like test-secret-2 already exists, because creation failed due to secrets &#34;test-secret-2&#34; already exists&#xA;Looks like test-secret-3 already exists, because creation failed due to secrets &#34;test-secret-3&#34; already exists&#xA;&lt; Exit [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/01/26 17:35:26.525 (58ms)&#xA;&gt; Enter [It] Upload critical/pipeline_with_artifact_upload_download.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/01/26 17:35:26.525&#xA;Uploading pipeline file critical/pipeline_with_artifact_upload_download.yaml&#xA;Creating temp pipeline file with overridden SDK Version&#xA;Uploading pipeline with name=e2e-test-1782927326467556723, from file /dspa/test_data/sdk_compiled_pipelines/valid/critical/pipeline_with_artifact_upload_download.yaml&#xA;Upload of pipeline file &#39;critical/pipeline_with_artifact_upload_download.yaml&#39; successful&#xA;Listing pipeline versions for pipeline 2f133d65-f1b1-4a6a-ba4e-3cf3bcb7cd7b&#xA;Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/critical/pipeline_with_artifact_upload_download.yaml&#xA;Returning pipeline run time inputs map[]&#xA;Create run for pipeline with id: &#39;2f133d65-f1b1-4a6a-ba4e-3cf3bcb7cd7b&#39; and name: &#39;e2e-test-1782927326467556723&#39;&#xA;Create a pipeline run for pipeline with id=2f133d65-f1b1-4a6a-ba4e-3cf3bcb7cd7b and versionId=465d4657-2039-499a-92f5-17833ef36d40&#xA;Create a pipeline run body&#xA;Created Pipeline Run successfully with runId=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Created Pipeline Run with id: aca9a53a-a842-4387-89a5-53d4af23cca0 for pipeline with id: 2f133d65-f1b1-4a6a-ba4e-3cf3bcb7cd7b&#xA;Waiting for pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 to be in one of &#39;[SUCCEEDED SKIPPED FAILED CANCELED]&#39;&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in PENDING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0 reached expected state SUCCEEDED&#xA;Deserializing expected compiled workflow file &#39;critical/pipeline_with_artifact_upload_download.yaml&#39; for the pipeline&#xA;Unmarshalling Expected Workflow YAML&#xA;Unmarshalled Expected Workflow YAML&#xA;Fetching updated pipeline run details for run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Get a pipeline run with id=aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Updated pipeline run details&#xA;Pipeline run succeeded, checking if the number of tasks are what is expected&#xA;&lt; Exit [It] Upload critical/pipeline_with_artifact_upload_download.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/01/26 17:37:46.927 (2m20.402s)&#xA;&gt; Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/01/26 17:37:46.927&#xA;################### Global Cleanup after each test #####################&#xA;Deleting 1 run(s)&#xA;Terminate run aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Failed to terminate run aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Terminate run aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Deleting run aca9a53a-a842-4387-89a5-53d4af23cca0&#xA;Deleting 0 experiment(s)&#xA;Deleting 1 pipeline(s)&#xA;Deleting pipeline with id=2f133d65-f1b1-4a6a-ba4e-3cf3bcb7cd7b (cascade=true)&#xA;Pipeline with id=2f133d65-f1b1-4a6a-ba4e-3cf3bcb7cd7b, DELETED&#xA;&lt; Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/01/26 17:37:47.072 (145ms)&#xA;&gt; Enter [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/01/26 17:37:47.073&#xA;&lt; Exit [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/01/26 17:37:47.073 (0s)&#xA;&gt; Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/01/26 17:37:47.073&#xA;&lt; Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/01/26 17:37:47.073 (0s)&#xA;</system-err>
          </testcase>
          <testcase name="[It] Upload and Verify Pipeline Run &gt; Upload a pipeline file, run it and verify that pipeline run succeeds Smoke &gt; Upload essential/iris_pipeline_compiled.yaml pipeline [FullRegression, smoke]" classname="AIPipelinesE2ETests" status="passed" time="195.843944181">
              <system-out>2026/07/01 17:38:42 Test passed&#xA;</system-out>
              <system-err>&gt; Enter [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/01/26 17:35:26.178&#xA;Create an experiment with name E2EExperiment-1782927326178929609&#xA;&lt; Exit [BeforeEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:147 @ 07/01/26 17:35:26.461 (283ms)&#xA;&gt; Enter [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/01/26 17:35:26.461&#xA;################### Setup before each Pipeline Upload test #####################&#xA;################### Global Setup before each test #####################&#xA;Test Context: 0x20d2cd344e00&#xA;test-secret-1 created&#xA;test-secret-2 created&#xA;test-secret-3 created&#xA;&lt; Exit [BeforeEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:49 @ 07/01/26 17:35:26.5 (38ms)&#xA;&gt; Enter [It] Upload essential/iris_pipeline_compiled.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/01/26 17:35:26.5&#xA;Uploading pipeline file essential/iris_pipeline_compiled.yaml&#xA;Creating temp pipeline file with overridden SDK Version&#xA;Uploading pipeline with name=e2e-test-1782927326461982332, from file /dspa/test_data/sdk_compiled_pipelines/valid/essential/iris_pipeline_compiled.yaml&#xA;Upload of pipeline file &#39;essential/iris_pipeline_compiled.yaml&#39; successful&#xA;Listing pipeline versions for pipeline e1f9c6af-1494-45c5-9d5d-5679f8506e2f&#xA;Get the pipeline run time inputs from pipeline spec file /dspa/test_data/sdk_compiled_pipelines/valid/essential/iris_pipeline_compiled.yaml&#xA;Returning pipeline run time inputs map[]&#xA;Create run for pipeline with id: &#39;e1f9c6af-1494-45c5-9d5d-5679f8506e2f&#39; and name: &#39;e2e-test-1782927326461982332&#39;&#xA;Create a pipeline run for pipeline with id=e1f9c6af-1494-45c5-9d5d-5679f8506e2f and versionId=b79a1a9c-d89d-4d10-b04d-49295c11adad&#xA;Create a pipeline run body&#xA;Created Pipeline Run successfully with runId=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Created Pipeline Run with id: 869b60b9-3c72-4856-8e00-1abc27f46578 for pipeline with id: e1f9c6af-1494-45c5-9d5d-5679f8506e2f&#xA;Waiting for pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 to be in one of &#39;[SUCCEEDED SKIPPED FAILED CANCELED]&#39;&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in PENDING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 is in RUNNING state, waiting...&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578 reached expected state SUCCEEDED&#xA;Deserializing expected compiled workflow file &#39;essential/iris_pipeline_compiled.yaml&#39; for the pipeline&#xA;Unmarshalling Expected Workflow YAML&#xA;Unmarshalled Expected Workflow YAML&#xA;Fetching updated pipeline run details for run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Get a pipeline run with id=869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Updated pipeline run details&#xA;Pipeline run succeeded, checking if the number of tasks are what is expected&#xA;&lt; Exit [It] Upload essential/iris_pipeline_compiled.yaml pipeline - /dspa/backend/test/end2end/pipeline_e2e_test.go:152 @ 07/01/26 17:38:41.925 (3m15.425s)&#xA;&gt; Enter [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/01/26 17:38:41.925&#xA;################### Global Cleanup after each test #####################&#xA;Deleting 1 run(s)&#xA;Terminate run 869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Failed to terminate run 869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Terminate run 869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Deleting run 869b60b9-3c72-4856-8e00-1abc27f46578&#xA;Deleting 0 experiment(s)&#xA;Deleting 1 pipeline(s)&#xA;Deleting pipeline with id=e1f9c6af-1494-45c5-9d5d-5679f8506e2f (cascade=true)&#xA;Pipeline with id=e1f9c6af-1494-45c5-9d5d-5679f8506e2f, DELETED&#xA;&lt; Exit [DeferCleanup (Each)] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:96 @ 07/01/26 17:38:42.022 (97ms)&#xA;&gt; Enter [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/01/26 17:38:42.023&#xA;&lt; Exit [ReportAfterEach] Upload and Verify Pipeline Run &gt; - /dspa/backend/test/end2end/pipeline_e2e_test.go:99 @ 07/01/26 17:38:42.023 (0s)&#xA;&gt; Enter [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/01/26 17:38:42.023&#xA;&lt; Exit [ReportAfterEach] TOP-LEVEL - /dspa/backend/test/end2end/e2e_suite_test.go:155 @ 07/01/26 17:38:42.023 (0s)&#xA;</system-err>
          </testcase>
      </testsuite>
  </testsuites>