Test Results Management and Analysis in WebUI Test Studio QA Edition

I started the series of blog posts on the new QA Edition of WebUI Test Studio 2010 with the Test List Management. In the current article I’m describing the Test Results Management and Analysis capabilities of the brand-new standalone tool.

The Result View

The test list execution in the QA Edition produces the objects we call Run Results. One can review and manage those results in the Results View:

The Results View in the Standalone represents the run results via RadScheduler. Once you execute your test list we switch to the Results View adding the new result into the Timeline View as in the image above. We also categorize the result by their pass and fail result (green vs. red items).

The Result View Commands

The next step is to select the result you are interested in. This will enable most of the commands in the Results View so that you can go ahead in the result management and analysis. The currently available commands are:

· Analyze Selected Run – you can go ahead and analyze the result (more on that later in this post).

· Clear All Results – the command will ask for confirmation and then delete all the available results both from the Scheduler and physically from the hard drive.

· Clear Selected Result – removes just the selected result.

· Result Publishing – this one requires Source Control connection as well as Visual Studio (either 2008 or 2010) to operate with MSTest. Once you have connected your test project to TFS you can choose the build you want to publish the selected result against. This improves the collaboration in the team – everyone can then go and check the functional test results directly in TFS having a better overview of the respective build quality. We may further explain and show this feature via a separate post.

· Export Result to Word and Excel – once you start analyzing the test results we offer the ability to export the result into Word or Excel files that can be shared with the management or the team folks.

· Back and Forward navigation – this eases the navigation between the different states on analyzing the test results.

Result Analysis

I’d like to focus the rest of this post mostly to the result analyzing capabilities in WebUI Test Studio QA Edition.

Once you select a run result you can click the Analyze Selected Run button (the first in the row) to trigger the command. An easy shortcut is to just double-click the run result you want to take a look. Both ways the Result View splits its layout loading the RunResult object in the Scheduler DayView on the left (we also navigate to the selected run result based on its start time). On the right side you get the test results of this run in a GridView along with the relevant information for those tests.

Each test result from the GridView can be further analyzed. Just double-click on a row and you get a step ahead. For example the first test from my example is a data-driven test running against google.com. When you step ahead in this result it will show you the two data iterations from which you can choose to drill-down further.

You also see the bread-crumbs like controls at the top of the TestResult pane – it helps figuring out the way you have drilled-down the data. In the example I have the first data iteration passed and the second failed for the first test in the run result.

Next I double-click one of the iteration and I get the test steps and each step result in particular. I prefer the failure one – it’s more interesting to find why this test has failed as opposite to find all the test steps in green.

So the test has failed because of the last verification – since I’ve data-driven the test, on execution the first test types the some value into the Google search input and then checks that same value appears in the results page input. The second test fails because we type some other value and then expect the old text to appear in the input. Of course the correct data-driven test should have both the action and verification data-driven in order to pass.

Let me focus more on the step failure details and resolution capabilities of WebUI Test Studio in a separate blog post.

Test Result Exporting

Any test result can be exported in the QA Edition once you drill-down to that data. For example from the above captured state I simply click on the Export to Excel command in the ribbon:

Once exported the test result can be shared as necessary. The above test result looks like this in Excel:

I hope this helps! Stay tuned for more updates of WebUI Test Studio.

Also, do not forget that the weekly "Get to know WebUI Test Studio" webinar is today at 11 AM US Eastern. This demo will provide you with an overview of the product capabilities and will show you how to create automated tests in less than 15 minutes. We'll be happy to answer all your questions on Telerik automated testing tools after the webinar.
You can register now here.

Yours,

-Konstantin