Integration test of Telerik Test Studio with Microsoft Tes Manager

16 posts, 1 answers
  1. José
    José avatar
    19 posts
    Member since:
    Mar 2013

    Posted 10 May 2013 Link to this post

    Hi,

    I have a doubt.
    I tried execute tests recorded in plugin of Telerik Test Studio for Visual Studio, 
    My doubt is, it's possible execute tests that was record in Telerik Test Studio in MTM(Microsoft Test Manager)? And if i'ts possible you can send me link of documentation or an explanation how to do it?

    best,

    José Teles
  2. Answer
    Velin Koychev
    Admin
    Velin Koychev avatar
    88 posts

    Posted 13 May 2013 Link to this post

    Hello José,

    Test Studio works nicely with Microsoft Test Manager. Our tests are functionally equivalent to Microsoft Coded UI Tests. You can easily run them from MTM. Anytime the Microsoft MSDN documentation refers to "Coded UI Test" you can substitute "Test Studio Test" in its place.

    A possible issue that you might experience is if you try to execute WPF tests.  You will need to hard code in the .tstest file the path to the WPF application. We have a feature request logged for this.

    I also attached a video example of MTM and VS2010.

    Please let me know whether the information is sufficient or if you have any additional questions.

    Greetings,
    Velin Koychev
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  3. David
    David avatar
    6 posts
    Member since:
    Oct 2013

    Posted 12 Nov 2013 Link to this post

    Velin,
    I am interested in seeing the video example you posted, but I am unable to play it back (audio is not working).  Do you have it in another format?   Is there any written documentation describing this integration?

    Cheers
    Dave   
  4. Velin Koychev
    Admin
    Velin Koychev avatar
    88 posts

    Posted 15 Nov 2013 Link to this post

    Hi Dave,

    You can find the same video also here

    Please start off by reviewing this video. While it doesn't talk about Microsoft Test Manager, it does give you an introduction how to integrate Test Studio tests into TFS. From there you need to launch MTM, create your test suite (which contains your test cases) and execute them. The MSDN documentation is pretty complete on how to do this. Functionally Test Studio tests work just like Visual Studio coded UI tests. If you follow the instructions on how to use coded UI tests in MTM and use Test Studio tests instead you should be just fine. The only thing you need special is that you must install either Test Studio full edition or our Run-Time edition on your Microsoft Test Controller and your Microsoft Test Agent machine (which may be the same machine).

    I hope this helps.

    Regards,
    Velin Koychev
    Telerik
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  5. David
    David avatar
    6 posts
    Member since:
    Oct 2013

    Posted 18 Nov 2013 Link to this post

    Velin,
    Can you just confirm if you video has audio or not?  (I am not hearing anything)  

    I have got to the point where my automation is associated with a test in Visual Studio.  I am able to see the steps in Visual Studio, but running it from there gives me the error shown in the attached picture.  Any Suggestions?

    Cheers
    Dave



      
  6. David
    David avatar
    6 posts
    Member since:
    Oct 2013

    Posted 18 Nov 2013 Link to this post

    Velin,
    A clarification on my last post.  The test will run if I press the start button circled in blue in the attached picture, pressing the button circled in red gives the error message previously posted.  Is this expected behavior?

    Cheers
    Dave 
  7. Daniel
    Daniel avatar
    13 posts
    Member since:
    Aug 2013

    Posted 18 Nov 2013 Link to this post

    The "Start" is the overall project build, which will fail because you technically do not have an executing project.

    You need to use Telerik green circle with the white play button, which will initiate the Telerik tests.
  8. David
    David avatar
    6 posts
    Member since:
    Oct 2013

    Posted 18 Nov 2013 Link to this post

    Hi Daniel,
    Thanks for the info.  
    What you say makes sense, but if I have to use the Telerick start button I have to open each test to get to it, which means I will end up running my tests one by one.  Can I use an executing project to effectively "wrap" my tests up and run them in one go?   If so, how do I set one up?

    Cheers
    Dave

  9. Daniel
    Daniel avatar
    13 posts
    Member since:
    Aug 2013

    Posted 18 Nov 2013 Link to this post

  10. David
    David avatar
    6 posts
    Member since:
    Oct 2013

    Posted 18 Nov 2013 Link to this post

    Hi Daniel,
    Another good point.  

    But how would the results be managed within Visual Studio?  Would there be a single pass/fail assigned against the containing test, or would the component tests be perceived as individual tests and the results managed accordingly?  I would imagine it was the first situation because all we are giving VS is one test, and there is no obvious reason why Visual Studio should be aware of the constituent tests.

    Cheers
    Dave



     
  11. Daniel
    Daniel avatar
    13 posts
    Member since:
    Aug 2013

    Posted 18 Nov 2013 Link to this post

    The first scenario is correct.  I run into this in my own projects all the time (as we can't use TS Test Lists in MTM or VS).

    Placing the code in the appropriate individual test should still allow the overall test to pass if the code kicks back off the same test and it passes.
  12. Velin Koychev
    Admin
    Velin Koychev avatar
    88 posts

    Posted 21 Nov 2013 Link to this post

    @ Daniel - Thank you for your help. We really appreciate it.  I have also updated your Telerik points.

    @ Dave - I confirm that this video does not have sound. Please let me know if you have any further questions. 

    Regards,
    Velin Koychev
    Telerik
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  13. David
    David avatar
    6 posts
    Member since:
    Oct 2013

    Posted 21 Nov 2013 Link to this post

    Hi Velin,
    Thanks for confirmation about the sound. 

    So to some up our experiences so far: 

    The integration is difficult to set up, there is very little documentation and instruction in this area.  After three days it's still not working for us and we are still seeing messages like  "File extension specified '.tstest' is not a valid test extension"  (Having said that, its only fair to say we are not just having problems with Telerick tests, we can't get coded UI tests to run across machines either)
     
    If the integration did work the differentiation of results at the test case level is not possible because the only way to run more than one test at a time is as tests within a test; the results coming back would be for the single container test  

    If the integration did work now, there is no guarantee it will work with Visual Studio 2013,  which is not expected to support Telerik Test Studio until mid Q1 next year.

    Is this a fair assessment?

    Cheers
    Dave
  14. Velin Koychev
    Admin
    Velin Koychev avatar
    88 posts

    Posted 25 Nov 2013 Link to this post

    Hello David,

    I agree that it might be difficult to properly configure this integration. I can see that you are currently discussing this problem with my colleague, Cody, in a support ticket. Did you try his suggestions?

    You are right that we currently do not support Visual Studio 2013. We hope that you will have preview for Christmas, but you are right, it will take awhile. 

    The Test Lists are deprecated in VS2012, but you can still use them.

    Here's the workaround that you can use, if you are working in VS 2012:

         1) Open your solution.
         2) From the Solution Explorer right click the solution item.
         3) Select Add/New Item
         4) Do not click on any of the available items, instead type in the text box 'TestList.vsmdi' or any other item name that ends with '.vsmdi'. Just make sure not to click on any item as it would overwrite your input.
         5) Find 'Solution Items' sub folder directly under your solution
         6) Double click TestList.vsmdi and this will bring up the 'Test List Editor' in VS2012.
     
    See this video for a demonstration. In VS2012 you can only create a test list and edit it, but in order to execute the test list you need to use MSTest via the command line.

    I hope this helps.

    Regards,
    Velin Koychev
    Telerik
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  15. David
    David avatar
    6 posts
    Member since:
    Oct 2013

    Posted 26 Nov 2013 Link to this post

    Hi Velin,
    I think Cody's has made it quite clear, that you guys are not experts in this area and his post was more of a clarification of terminology than a solution.  His main point is, if we want to resolve our issues we should be looking more toward Microsoft rather than Telerik.  I understand this I will be happy to explore further, but I was hoping for a more "out of the box" solution - time is not on our side.

    Thanks for the info on Test Lists, but is this really the way forward if this functionality is deprecated? I am concerned about investing time and effort in a technology that is effectively defunct.

    Cheers
    Dave


      
  16. Velin Koychev
    Admin
    Velin Koychev avatar
    88 posts

    Posted 29 Nov 2013 Link to this post

    Hello Dave,

    I understand your concerns.
    Only in time we will know whether Microsoft took the right decision to deprecate test lists.  In the mean time, unfortunately this is all we can currently offer you. 

    Thank you for your understanding.

    Regards,
    Velin Koychev
    Telerik
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
Back to Top