Integrating your Windows Phone unit tests with MSBuild #6: Conclusion

It’s been a while since I started this blog post series which describes our approach of integrating and automating the unit test execution with MSBuild thus effectively organizing a Continuous Integration build which aggregates check-ins, builds the projects and runs all unit tests after that. Since there is no out-of-the-box solution to automate Windows Phone unit tests we came up with some not so standard ideas and managed to build our own mechanism. I have described these ideas in a series of blogs summarized below:

1. Introduction and Overview – description of the solution, as well as some explanations of the reasoning behind this approach
2. The Test Runner Task – the topic here is the custom MSBuild task, part of the automation mechanism
3. The Unit Test Application – here are discussed the details about the application that runs the unit tests
4. The Web Service – the communication channel between the Test Runner Task and the Unit Test Application and its implementation

In general, we have implemented a mechanism that helps us automate the execution of our Windows Phone 7 unit tests by integrating it with MSBuild. This this way, we do not need to run them manually since our check-ins trigger their execution thus constantly providing us with information about the stage of our codebase. The idea behind the mechanism is quite simple: we have a custom MSBuild task that is executed after the solution containing the unit tests is built. This task takes the ready .XAP file representing the unit test application (which is a project in the solution), installs it on the Windows Phone 7 emulator and runs it. The unit test application on the other side logs information about each unit test’s execution and at the end stores the data on the build machine. The MSBuild task finds the results and returns them back to the developer by integrating them into the final build log. The scheme below summarizes the whole approach:

The implementation of this mechanism was not effortless and introduced some challenges to us. It came out that there is not a straightforward way to perform I/O operations with the IsolatedStorage of an application installed on the Windows Phone 7 emulator by using the automation API. Therefore we came up with the idea to introduce a web service and use it as an interface between the app on the phone and the MSBuild task. This approach, on the other side, is not easily debugged (due to the fact that a couple of components play together) but once understood and set up it proved to be easily supported and serviced.

With this series I hope that I have given a good example of a possible way to automate Windows Phone 7 unit tests. If you have different ideas or approaches in mind we would be happy to hear from you and discuss possible improvement options.