Test Studio integration with CI (TeamCity)

[Josh]

I have been trying to explore the integrating Test Studio with our current CI (Team City 7.x).

I have been reviewing the existing documentation here.

The type of testing we are wanting to perform would be automated testing of a WPF application.

My question is, is there a way I can set this up to run headless. i.e. Not have a user logged in/remoted in to the machine with the build runner on it?

Thanks in advance for any help.

Josh

[Byron]

Hi Josh,

Unfortunately, it will not be possible to automate against a WPF application on a remote machine with no active session. WPF automation is desktop automation, and desktop automation requires an active session. 

Greetings,
Byron
the Telerik team

[Josh]

Byron,

Thanks for the fast reply. I have just one more question. Apparently we have a Scheduling Server installed on our network. Would it be possible for me to schedule tests to be run from our CI on that server? 

The reason I ask is that while I have control over the CI, the Scheduling Server is being maintained by someone else. If I could just schedule the jobs, then they could work out the active session issue.

Thanks again,
Josh

[Byron]

Hello Josh,

Unfortunately not. CI servers have two options for executing Test Studio tests: tests can be run with ArtOfTest.Runner on the command line, or with MSTest. Either of these runners can execute tests and test lists and publish results, and ArtOfTest.Runner even has an option to publish test run results to a scheduling server results database. But neither has an option to schedule a test with a scheduling server; this requires a Full Test Studio license and access to the Test Studio client. That being said, you could schedule your test using the Test Studio scheduling server, but this would not be triggered by your CI server triggers, such as check-ins. Of course, these distinctions are largely academic: regardless of how you schedule or trigger a Test Studio test execution, the execution machine will require an active session to perform desktop automation.

Kind regards,
Byron
the Telerik team

[Josh]

Byron,

Got it. Thanks again for all your help and clarifications.

Josh

[Moss]

Hi Byron,

Could you please give a sample MSTest command line to execute Test Studio Test lists on TeamCity build?
I assume this is achievable as you mentioned in the previous post.

Many Thanks,
Moss

[Rodney]

Hello Moss,

Byron is unavailable so allow me to interject. You can't execute Test Studio test lists with MSTest. If you need to execute Test Studio test lists, then you should use ArtofTest.Runner.

MSTest will let you run lists created in Visual Studio Test List Editor. Here’s a knowledge base article that outlines some of the more common execution scenarios: Executing Tests with MSTest.

For additional MSTest commands, I recommend having a look at MSTest.exe Command Line Options.

Regards,
Rodney
Telerik

[Arnar]

How is it possible to run UI tests with ArtOfTest.Runner on the command line using TeamCity? I am currently having problem with running the tests in the CI build, the following error is ArtOfTest.InternetExplorer assembly missing.. Do you know any possible solution?

 

[Cody]

Hello Arnar,

I answered this question in your private support ticket. Did you find that information useful? I haven't heard back from you yet.


Regards,
Cody
Telerik

 

[Esala]

Nightly basis, build server (TeamCity) will build everything on a release branch. And I want to trigger entire test suite if build got successful. Can this be done using Test Studio? If you can you please point me the location that has this information? (Documentation, video etc)

[Elena]

Hi Esala,

Please find here further details how you could integrate TeamCity with Test Studio. I hope this information will be useful to you. 

Regards,
Elena Tsvetkova
Telerik by Progress