Telerik blogs

Integrating your Windows Phone unit tests with MSBuild #3: The dedicated Test Runner Task

Deyan Ginev by Deyan Ginev
January 13, 2011 Mobile 0 Comments


First of all, let me wish you a very happy and productive 2011! May the good Silverlight forces be with you and let’s hope that this year will be full of technology excitements and challenges!

For the ones who haven’t followed me until now – this is the third installment of a blog post series focusing on an automation infrastructure for the execution of unit tests for Windows Phone 7 with MSBuild. This blog focuses on the MSBuild task, part of the unit test automation infrastructure, that installs the unit test application on the Windows Phone 7 emulator and runs the tests. I am going to explain the way the build task functions by commenting on parts of the source code. If you need to take a look at the previous blog posts from the series you can find them here.

The backbone of the Unit Test Automation Infrastructure that we have been discussing is a dedicated MSBuild task. This task does the job to get the built unit test application, install it on the WP7 emulator and start the unit test execution. The task also reads the test results and integrates them into the final MSBuild log that you will see in Visual Studio’s Team explorer. If you need additional information on how a custom MSBuild task is created, you can take a look at this article on MSDN: http://msdn.microsoft.com/en-us/library/t9883dzc.aspx

Assuming that you are familiar with the way custom MSBuild tasks are created I will go on with the source code of the Execute method of the custom test runner task:

 

public override bool Execute()
{
     if (!this.PerformChecksAndInitialization())
     {
          return false;
     }
  
     this.CleanUpPreviousTestsResults();
  
     WP7RemoteDeviceManager remoteDevManager = new WP7RemoteDeviceManager(1033);
     WP7Emulator emulator = new WP7Emulator(remoteDevManager);
  
     WP7ApplicationInfo appInfo = this.GenerateTestApplicationInfo();
     this.deployedApplication = this.DeployApplication(emulator, appInfo);
  
     if (this.deployedApplication == null)
     {
         emulator.Disconnect();
         this.LogError("Error occured while loading the test application.");
         return false;
     }
  
     this.testResults = this.LaunchAndWaitForResult(emulator, this.deployedApplication);
  
     this.deployedApplication = null;
  
     if (this.testResults == null)
     {
           this.LogError("Could not retrieve test results. Test run failed.");
           return false;
     }
  
     return this.InterpretTestResults(this.testResults);
}

 

As I mentioned earlier, the Microsoft.SmartDevice.Connectivity API is utilized to install and run applications on the WP7 emulator whereby I have wrapped this functionality in a couple of classes for my convenience. The implementation of these classes, as well as the relationships between them, are not important. The names of the methods used are descriptive enough and thus I am not going to dive deeper into these wrapper classes.

As you can see from the source code the first thing to do is install the unit test application on the WP7 emulator. I am using my wrapper classes (WP7DeviceManager and WP7Emulator) to get an instance of the WP7 emulator. After that I generate an Application Info object (again part of the wrappers) which holds information about the .XAP file that is to be installed on the emulator. The real deal here happens in the DeployApplication method:

private WP7Application DeployApplication(WP7Emulator device, WP7ApplicationInfo appInfo)
{
      try
      {
          if (!device.IsConnected())
          {
              device.Connect();
          }
  
          If the application is already installed, remove the existing instance
          if (device.IsApplicationInstalled(appInfo))
          {
               WP7Application existingApp = device.GetApplication(appInfo);
               if (existingApp != null)
               {
                   device.StopApplication(existingApp);
                   device.UninstallApplication(existingApp);
               }
          }
  
          WP7Application installedApp = device.InstallApplication(appInfo);
          return installedApp;
      }
      catch (Exception e)
      {
           this.LogError("An error occured while deploying the test application on the target device: " + e.Message);
           device.Disconnect();
      }
  
      return null;
}

 

In this method the emulator is activated by calling the Connect method. After that, a check is performed whether a previous version of the application is already installed on the emulator. If yes, it is uninstalled and the fresh built app is uploaded. And this is what the DeployApplication routine does.
The next step is to run the unit test application and read the results. Here comes the LaunchAndWaitForResult method shown in the listing below:

private TestRunInfo LaunchAndWaitForResult(WP7Emulator emulator, WP7Application app)
{
    if (this.testAutomationServ == null)
    {
        this.testAutomationServ = new WPCommunicationServiceSoapClient();
    }
    this.LogMessage("Starting test application...");
    emulator.LaunchApplication(app);
    this.LogMessage("Test application started...");
  
    if (this.BuildEngine != null)
    {
        BuildMessageEventArgs args = new BuildMessageEventArgs(
            "Test run started...",
            "Test",
            typeof(RunTestsTask).Name,
            MessageImportance.Normal,
            DateTime.UtcNow
            );
  
        this.BuildEngine.LogMessageEvent(args);
    }
  
    TestRunInfo testResults = null;
    string key = this.GenerateTestSourceIdentificationKey();
    DateTime taskTimeout = DateTime.Now;
    int taskTimeoutTime = this.TaskTimeout;
    this.LogMessage("Waiting for test results...");
    while (testResults == null)
    {
        Thread.Sleep(this.PollingInterval * 1000);
  
        //The test application exited, now we can read the results.
        testResults = this.testAutomationServ.ReadTestResults(key);
  
        if ((DateTime.Now - taskTimeout).TotalSeconds > taskTimeoutTime)
        {
            this.LogError("The test timeout.");
            break;
        }
    }
    return testResults;
}

 

This method takes a WP7 emulator instance and the unit test application object as arguments. The method launches the application and starts waiting for results.  Since - as mentioned in the previous blog post of the series – there is now direct way to check whether an application is running on the WP7 emulator, there is a while loop that constantly polls for results by calling a web service which is used to store and read test execution data. This web service will be discussed in the next post but what you currently need to now is that it is used both by the unit test application and the MSBuild task to store/read test results.

So, when the test results are available, the while loop exits and we can integrate the information with the final MSBuild log that you can see from within Visual Studio’s Build Log viewer. This integration happens in the InterpretTestResults method listed below:

 

private bool InterpretTestResults(TestRunInfo results)
{
    bool result = results.IsSuccess;
  
    if (result && this.BuildEngine != null)
    {
        BuildMessageEventArgs args = new BuildMessageEventArgs(
        "Test run completed successfully.",
        "Test",
        typeof(RunTestsTask).Name,
        MessageImportance.Normal,
        DateTime.UtcNow
        );
  
        this.BuildEngine.LogMessageEvent(args);
  
        return true;
    }
  
    StringBuilder failedInfo = new StringBuilder();
    foreach (TestMethodInfo info in this.testResults.TestMethodInfos)
    {
        if (info.IsSuccess)
        {
            continue;
        }
  
        string testFailedInfo = string.Format("Test {0} in {1} failed with: {2}", info.TestName, info.TestClassName, info.FailInfo);
        failedInfo.Append(testFailedInfo + Environment.NewLine);
    }
  
    this.LogError("Test run failed:" + Environment.NewLine + failedInfo.ToString());
  
    return result;
}

 

 

This method uses the TestRunInfo type which is nothing more than a class holding information about each unit test execution. The InterpretTestResults class iterates over all TestMethodInfo objects (holding information about a single unit test execution) and if there is a failure, the information about the test and the type of failure is logged.

So this is the skeleton of the dedicated MSBuild task that performs a significant part of the MSBuild Unit Test Automation process for our Windows Phone 7 unit tests. I believe the most important details are addressed but you can feel free to post your questions and demand for additional explanations.
Happy coding!

UI, Windows
Deyan Ginev
About the Author

Deyan Ginev

Deyan has been with Telerik for more than ​six years working on several different products with main focus on mobile technologies. Deyan is currently working on NativeScript– a platform for developing native mobile apps with JavaScript. Find him on Twitter via
@deyan_ginev.

Comments

Comments are disabled in preview mode.