This is a migrated thread and some comments may be shown as answers.

Automatic re-run of Failed tests from a Test List execution

0 Answers 440 Views
General Discussions
This is a migrated thread and some comments may be shown as answers.
This question is locked. New answers and comments are not allowed.
Daniel
Top achievements
Rank 1
Daniel asked on 11 Nov 2014, 09:39 AM
Here is a solution how to perform an automatic re-run of failed tests after a Test List execution finishes. This is done by using the exposed functionality by Test Studio Execution Extension (http://docs.telerik.com/teststudio/user-guide/code-samples/general/execution-extensions.aspx).

Note: This is just an ready-to-use working example to see how this can be achieved, but you can change it in any way and just reuse the concept with your custom implementation (attached is the solution for building the extension, in which to make the customizations).

1. Open the attached execution extension archive (ReRunFailedTestsExecutionExtension.zip) which contains already built execution extension with .Net 4.5

2. The code inside overwrites the default methods OnBeforeTestListStarted(), OnBeforeTestStarted() and OnAfterTestListCompleted() to build a list of failed tests from the Test List result object for later use for the re-run functionality

3. Copy/Paste the ReRunFailedTests.dll and ReRunFailedTests.dll.config files in your [Test Studio installation folder]\Bin\Plugins\ folder so they will be used whenever Test Studio or command line ArtOfTest.Runner.exe are started

4. In the configuration file ReRunFailedTests.dll.config there are several keys that can be changed:
4.1. 'ReRun = True' means the automatic re-run functionality will be triggered for each test list run. If you don't want it to be triggered set it to False
4.2 'ReRunAsList = True' option will execute all failed tests as part of a single new Test List created from the original Test List, but containing only the failed tests from the original list. The 'name' of the new list will be the same as the original list and added (FailedReRun) at the end of it; if 'ReRunAsList' key is set to 'False' the automatic rerun of all failed tests will be performed on test by test basis, meaning failed tests will not be combined in a Test List and executed as tests - this will effect in generating an individual result for each failed test, i.e. if your test list has 100 tests and 10 of them fail, after the rerun you will have single result from the original Test List run and additionally 10 individual small results for each failed test (if using 'True' option you will only have 2 result files in this example, instead of 11 to be shown in Test Studio Results view);
4.3 'ProcessToRun' is the path to where ArtOfTest.Runner.exe is installed (comes with default path installation on x64 machine)

Note1: in order for this approach to work the tests should be written as autonomous tests - created in a way to be executed on their own, without the use of previous tests run before them in a list to make them work properly
Note2: this will work only for local test list executions, not in a remote/scheduled runs on distributed machines
Tags
General Discussions
Asked by
Daniel
Top achievements
Rank 1
Share this question
or