How to run my tests in defined order

[ching ping]

Hi Telerik Team,

I'm using WebUI Test Studio Developer edition to create my UI tests. Currently, i'm adding my tests into Test List and running my tests with MSTest with vsmdi file. How can i set my tests to run in specified order? I understand that QA edition do have feature that set tests to run in specified order. How can i achieve that in dev edition?

[Cody]

Hello ching ping,

What you can do is create an "Ordered Test". This is a Microsoft test type which simply accepts a list of other tests and runs them in the specific order you specify. Then you instruct MSTest to execute your Ordered Test, by name, and it will begin running the tests you put into your Ordered Test.

All the best,
Cody

the Telerik team

[Ibrahim Imam]

hello

i had the same issue and also used an ordered test (as i read this answer on another forum)
i am using master and subtests so some of my tests i add to the ordered test are testfragments (having the property set to true so they don't run by themselves). unfortunately when i start the ordered test it fails.
when using the usual test list and add the tests (including testfragment so they exist in output directory) it works correctly, the tests pass.
i add three tests, one of them is a testfragment. the test summary of ordered test says:
Failed 1 (this i the ordered test)
Passed 2 (this are the two tests)
Not runnable 1 (this is the test fragment)

is there any way to make the ordered test work with test fragments or can i always copy all tests in output folder so i don't have to add the test fragment to ordered test?

best regards

[Cody]

Hello ching ping,

Your test fragments should not be included as tests in your Ordered test list. Instead you should add them as deployment items as shown in the attached screenshot. This will copy them to the Out folder, making them available for use by the main tests, but Visual Studio won't try to execute them as if they were a main test since they weren't included in the Ordered test list.

Kind regards,
Cody

the Telerik team

[Karen]

Hi,

I am using the ArtOfTest.Runner to execute my test lists from the standalone version. Is there a way to order my tests without using VS Ordered Test Lists?

Thanks,

Karen

[Cody]

Hello Karen,

Our command line runner can run individual Test Studio tests and Test Studio Test Lists. It is not able to execute VS Orders Test Lists. Only MSTest can do that. You would have to obtain a license of our full Test Studio before you would be able to create Test Studio Test Lists. Until you obtain this license your only option with ArtOfTest.Runner.exe is to tell it to run individual tests, one test at a time, in a specific order. Perhaps using a .cmd file.

All the best,
Cody
the Telerik team

[Karen]

Hi Cody,

Sorry, my previous post didn't have much context.

I actually have a trial license for Test Studio and am currently trying to decide if it will meet our functional testing needs.

We are using ArtOfTest.Runner because we don't want to install MSTest on our Team City build agent. I have already created a dynamic Test Studio Test List and am running it from the command line. This is currently working great, minus the ability to specify test order.

I did notice that the test lists run the tests in alphabetical order, and am aware of the option to alphabetize my test names. Unfortunately, this is a non-elegant solution, so I was hoping there would be something built in already.

Hope this clarifies my situation.

Karen

[Cody]

Hello,

Thank you for the clarification. There is no way to specify the execution order of dynamic test lists. Instead you need to use a static test list. Then you'll be able to control the exact execution order you want.

All the best,
Cody
the Telerik team