Integration test of Telerik Test Studio with Microsoft Tes Manager

[José]

Hi,

I have a doubt.
I tried execute tests recorded in plugin of Telerik Test Studio for Visual Studio, 
My doubt is, it's possible execute tests that was record in Telerik Test Studio in MTM(Microsoft Test Manager)? And if i'ts possible you can send me link of documentation or an explanation how to do it?

best,

José Teles

[Velin Koychev]

Hello José,

Test Studio works nicely with Microsoft Test Manager. Our tests are functionally equivalent to Microsoft Coded UI Tests. You can easily run them from MTM. Anytime the Microsoft MSDN documentation refers to "Coded UI Test" you can substitute "Test Studio Test" in its place.

A possible issue that you might experience is if you try to execute WPF tests.  You will need to hard code in the .tstest file the path to the WPF application. We have a feature request logged for this.

I also attached a video example of MTM and VS2010.

Please let me know whether the information is sufficient or if you have any additional questions.

Greetings,
Velin Koychev
the Telerik team

[David]

Velin,
I am interested in seeing the video example you posted, but I am unable to play it back (audio is not working).  Do you have it in another format?   Is there any written documentation describing this integration?

Cheers
Dave   

[Velin Koychev]

Hi Dave,

You can find the same video also here. 

Please start off by reviewing this video. While it doesn't talk about Microsoft Test Manager, it does give you an introduction how to integrate Test Studio tests into TFS. From there you need to launch MTM, create your test suite (which contains your test cases) and execute them. The MSDN documentation is pretty complete on how to do this. Functionally Test Studio tests work just like Visual Studio coded UI tests. If you follow the instructions on how to use coded UI tests in MTM and use Test Studio tests instead you should be just fine. The only thing you need special is that you must install either Test Studio full edition or our Run-Time edition on your Microsoft Test Controller and your Microsoft Test Agent machine (which may be the same machine).

I hope this helps.

Regards,
Velin Koychev
Telerik

[David]

Velin,
Can you just confirm if you video has audio or not?  (I am not hearing anything)  

I have got to the point where my automation is associated with a test in Visual Studio.  I am able to see the steps in Visual Studio, but running it from there gives me the error shown in the attached picture.  Any Suggestions?

Cheers
Dave



  

[David]

Velin,
A clarification on my last post.  The test will run if I press the start button circled in blue in the attached picture, pressing the button circled in red gives the error message previously posted.  Is this expected behavior?

Cheers
Dave 

[Daniel]

The "Start" is the overall project build, which will fail because you technically do not have an executing project.

You need to use Telerik green circle with the white play button, which will initiate the Telerik tests.

[David]

Hi Daniel,
Thanks for the info.  
What you say makes sense, but if I have to use the Telerick start button I have to open each test to get to it, which means I will end up running my tests one by one.  Can I use an executing project to effectively "wrap" my tests up and run them in one go?   If so, how do I set one up?

Cheers
Dave

[Daniel]

You could create a test and them run all the other tests as Steps (http://www.telerik.com/automated-testing-tools/support/documentation/user-guide/add-custom-step/test-as-step.aspx).

[David]

Hi Daniel,
Another good point.  

But how would the results be managed within Visual Studio?  Would there be a single pass/fail assigned against the containing test, or would the component tests be perceived as individual tests and the results managed accordingly?  I would imagine it was the first situation because all we are giving VS is one test, and there is no obvious reason why Visual Studio should be aware of the constituent tests.

Cheers
Dave



 

[Daniel]

The first scenario is correct.  I run into this in my own projects all the time (as we can't use TS Test Lists in MTM or VS).

Placing the code in the appropriate individual test should still allow the overall test to pass if the code kicks back off the same test and it passes.

[Velin Koychev]

@ Daniel - Thank you for your help. We really appreciate it.  I have also updated your Telerik points.

@ Dave - I confirm that this video does not have sound. Please let me know if you have any further questions. 

Regards,
Velin Koychev
Telerik

[David]

Hi Velin,
Thanks for confirmation about the sound. 

So to some up our experiences so far: 

The integration is difficult to set up, there is very little documentation and instruction in this area.  After three days it's still not working for us and we are still seeing messages like  "File extension specified '.tstest' is not a valid test extension"  (Having said that, its only fair to say we are not just having problems with Telerick tests, we can't get coded UI tests to run across machines either)
 
If the integration did work the differentiation of results at the test case level is not possible because the only way to run more than one test at a time is as tests within a test; the results coming back would be for the single container test  

If the integration did work now, there is no guarantee it will work with Visual Studio 2013,  which is not expected to support Telerik Test Studio until mid Q1 next year.

Is this a fair assessment?

Cheers
Dave

[Velin Koychev]

Hello David,

I agree that it might be difficult to properly configure this integration. I can see that you are currently discussing this problem with my colleague, Cody, in a support ticket. Did you try his suggestions?

You are right that we currently do not support Visual Studio 2013. We hope that you will have preview for Christmas, but you are right, it will take awhile. 

The Test Lists are deprecated in VS2012, but you can still use them.

Here's the workaround that you can use, if you are working in VS 2012:

     1) Open your solution.
     2) From the Solution Explorer right click the solution item.
     3) Select Add/New Item
     4) Do not click on any of the available items, instead type in the text box 'TestList.vsmdi' or any other item name that ends with '.vsmdi'. Just make sure not to click on any item as it would overwrite your input.
     5) Find 'Solution Items' sub folder directly under your solution
     6) Double click TestList.vsmdi and this will bring up the 'Test List Editor' in VS2012.
 
See this video for a demonstration. In VS2012 you can only create a test list and edit it, but in order to execute the test list you need to use MSTest via the command line.

I hope this helps.

Regards,
Velin Koychev
Telerik

[David]

Hi Velin,
I think Cody's has made it quite clear, that you guys are not experts in this area and his post was more of a clarification of terminology than a solution.  His main point is, if we want to resolve our issues we should be looking more toward Microsoft rather than Telerik.  I understand this I will be happy to explore further, but I was hoping for a more "out of the box" solution - time is not on our side.

Thanks for the info on Test Lists, but is this really the way forward if this functionality is deprecated? I am concerned about investing time and effort in a technology that is effectively defunct.

Cheers
Dave


  

[Velin Koychev]

Hello Dave,

I understand your concerns.
Only in time we will know whether Microsoft took the right decision to deprecate test lists.  In the mean time, unfortunately this is all we can currently offer you. 

Thank you for your understanding.

Regards,
Velin Koychev
Telerik