How to run my tests in defined order

8 posts, 0 answers
  1. ching ping
    ching ping avatar
    3 posts
    Member since:
    Sep 2010

    Posted 21 Nov 2010 Link to this post

    Hi Telerik Team,

    I'm using WebUI Test Studio Developer edition to create my UI tests. Currently, i'm adding my tests into Test List and running my tests with MSTest with vsmdi file. How can i set my tests to run in specified order? I understand that QA edition do have feature that set tests to run in specified order. How can i achieve that in dev edition?
  2. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 22 Nov 2010 Link to this post

    Hello ching ping,

    What you can do is create an "Ordered Test". This is a Microsoft test type which simply accepts a list of other tests and runs them in the specific order you specify. Then you instruct MSTest to execute your Ordered Test, by name, and it will begin running the tests you put into your Ordered Test.

    All the best,
    Cody
    the Telerik team
    See What’s New in WebUI Test Studio’s Q3 Release 24/7 on Telerik TV
  3. Ibrahim Imam
    Ibrahim Imam avatar
    69 posts
    Member since:
    Jul 2007

    Posted 30 Dec 2010 Link to this post

    hello

    i had the same issue and also used an ordered test (as i read this answer on another forum)
    i am using master and subtests so some of my tests i add to the ordered test are testfragments (having the property set to true so they don't run by themselves). unfortunately when i start the ordered test it fails.
    when using the usual test list and add the tests (including testfragment so they exist in output directory) it works correctly, the tests pass.
    i add three tests, one of them is a testfragment. the test summary of ordered test says:
    Failed 1 (this i the ordered test)
    Passed 2 (this are the two tests)
    Not runnable 1 (this is the test fragment)

    is there any way to make the ordered test work with test fragments or can i always copy all tests in output folder so i don't have to add the test fragment to ordered test?

    best regards
  4. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 03 Jan 2011 Link to this post

    Hello ching ping,

    Your test fragments should not be included as tests in your Ordered test list. Instead you should add them as deployment items as shown in the attached screenshot. This will copy them to the Out folder, making them available for use by the main tests, but Visual Studio won't try to execute them as if they were a main test since they weren't included in the Ordered test list.

    Kind regards,
    Cody
    the Telerik team
    Interested in Agile Testing? Check out Telerik TV for a recording of Automated Testing in the Agile Environment
  5. Karen
    Karen avatar
    4 posts
    Member since:
    Jul 2012

    Posted 06 Aug 2012 Link to this post

    Hi,

    I am using the ArtOfTest.Runner to execute my test lists from the standalone version. Is there a way to order my tests without using VS Ordered Test Lists?

    Thanks,

    Karen
  6. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 06 Aug 2012 Link to this post

    Hello Karen,

    Our command line runner can run individual Test Studio tests and Test Studio Test Lists. It is not able to execute VS Orders Test Lists. Only MSTest can do that. You would have to obtain a license of our full Test Studio before you would be able to create Test Studio Test Lists. Until you obtain this license your only option with ArtOfTest.Runner.exe is to tell it to run individual tests, one test at a time, in a specific order. Perhaps using a .cmd file.

    All the best,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  7. Karen
    Karen avatar
    4 posts
    Member since:
    Jul 2012

    Posted 06 Aug 2012 Link to this post

    Hi Cody,

    Sorry, my previous post didn't have much context.

    I actually have a trial license for Test Studio and am currently trying to decide if it will meet our functional testing needs.

    We are using ArtOfTest.Runner because we don't want to install MSTest on our Team City build agent. I have already created a dynamic Test Studio Test List and am running it from the command line. This is currently working great, minus the ability to specify test order.

    I did notice that the test lists run the tests in alphabetical order, and am aware of the option to alphabetize my test names. Unfortunately, this is a non-elegant solution, so I was hoping there would be something built in already.

    Hope this clarifies my situation.

    Karen
  8. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 06 Aug 2012 Link to this post

    Hello,

    Thank you for the clarification. There is no way to specify the execution order of dynamic test lists. Instead you need to use a static test list. Then you'll be able to control the exact execution order you want.

    All the best,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
Back to Top
  Cast Your Vote at DevPro Connections! Cast Your Vote at  Win IT Pro Connections!