Test list - How to stop on failure

[Daniel]

Hey guys,

currently we're trying to integrate our tests in azure devops pipeline. We want to achieve that the testlist we're running will stop on failure. How can we do that?

I've found the documentation here but it only describes the solution for the desktop application. Not if we want to achieve this behavior in Visual Studio or in Azure Devops pipeline.

Thanks and regards,

Daniel

[Missing User]

Hi Daniel,

There is another option to stop the test list execution on test failure, but again it is applicable only per test. 

In order to achieve above for all your test you can extend the BaseWebAiiTest as described in this article. In the extended class you can override the method OnBeforeTestStarted(BeforeTestStartedArgs args) and set the property StopTestListOnFailure.

Hereafter you have to make sure all your existing code behind files (tstest.cs) inherit your extended class (ExtendedBaseWebAiiTest) instead of the BaseWebAiiTest.

   public class ExtendedBaseWebAiiTest : BaseWebAiiTest
    {
        public override void OnBeforeTestStarted(BeforeTestStartedArgs args)
        {
            args.Test.StopTestListOnFailure = false;
            base.OnBeforeTestStarted(args);
        }
    }

 

In order project to compile you have to do one more thing - add the following two references in the project Settings.aii file in the ProjectReferences array: 

"System.Runtime.Serialization",
"Telerik.TestStudio.Interfaces"

Should you have any additional questions, please let me know.

Regards,
Stoyan
Progress Telerik

[Daniel]

Thank you very much for the idea, I'm already using such an extended base class so the implementation is pretty good in my case!

[Missing User]

Hi Daniel,

I'm glad that we've found the solution that suits you the most.

[Missing User]

Hello Daniel,

I'm sorry to hear you experience troubles with the setup of the Azure pipeline and will assist you with sorting this out.

From the description bellow, I'm assuming that you are using either Test Studio plugin or Testing Framework to run the tests in Visual Studio or Azure Devops.

If you are using Visual Studio plugin, you can edit the test settings and enable StopTestListOnFailure  property as described here.

If you are using Telerik Testing Framework, you can set  StopTestListOnFailure  property directly in the Test object, here you can find the API reference.

I hope the above suggestions will work for you. If you have any further questions don't hesitate to contact Test Studio Support team.

Regards,
Stoyan
Progress Telerik

[Daniel]

Since we're developing our (coded) tests (.tstest.cs files) in Visual Studio, I assume we're using the Test Studio plugin. In Azure Devops we're triggering after deploying our code the "Progress\Test Studio\Bin\ArtOfTest.Runner.exe" to run our coded tests. If there is any other possible way to run our tests automated with continious integration, give us a notice.

The way to edit the test settings aren't our use case, we want to define inside a test list to stop triggering the other tests if one already failed.