Test Scheduling Updates in 2010 Q3 Service Pack 1

This week we published the latest official release of our automated testing tools – 2010 Q3 Service Pack. This release enhances product stability by introducing a lot of improvements and bug fixes in many areas. I’d like to focus this post on the latest optimizations and important bug fixes in the Test Scheduling feature.

I’ll start with the QA Edition changes.

Project Tab

Now you can find the following Scheduling commands in the Project Tab.

The project tab allows you to:

1. Connect the Project to the Scheduling Server.

In order to schedule a TestList for execution the project has to be connected to a Scheduling Server. See the Scheduling FAQ for details and also check this video to learn how to perform this step during the setup.

As soon as the project is connected the user can manage the execution machines.

2. View/manage registered machines.

The “Machines” button will open the Machine Viewer.

The user can activate and deactivate the machines in the network that have been registered as execution servers. The Machine Viewer enables users to manually (de)activate a machine from the QA Edition instead of having to log on to that machine.

TestLists Tab

The Schedule TestList command there allows the user to schedule a TestList for local or remote execution. For details please check this video.

Results Tab

The Scheduling Server buttons give the user the abilities to load, publish and manage test run results:

1. Reload Results from Server – retrieves completed test run results from the server to display locally on the results calendar.

2. Publish Results to Server – this new command allows the user to send local results to the server.

3. Manage/View Results – This button launches our new ‘Run Results Viewer’ which displays information about the specific RunResult such as name, execution machine, start and end time as well as summary (passed/fails test results). One can also use the option to delete a run result from the server if it’s not needed anymore – this will remove the information about that result from the scheduling server database.

I’d like to share some more test scheduling options outside the QA Edition.

Configure Scheduling Server

Those who have set up the Scheduling Server running the previous release will notice a small but useful improvement here. In the DB Connection Settings we’ve added a dropdown that lists the available SQL Servers in the network.

Note this heavily depends on the network responsiveness. If the network responds fine, one will get all the servers available for connection, otherwise some may be missing. The user can still manually type the server he/she needs to connect to.

The other option I’d like to mention is the ability to change the timer tick period when configuring the server:

Let me elaborate on what this is and when it makes sense to use it.

Once the Scheduling Server gets configured we run a timer that ticks on certain interval. Each time it ticks, we perform a search in the database to see if there is a TestList that has been scheduled for execution and that needs to be started. This setting adjusts the frequency that the server is checked for scheduled test lists – if one schedules a TestList for execution at 1:00 a.m. and the timer ticks away at 1:02, the execution will start then.

The default timer tick period is 5 minutes and with this release you can change it from the UI to fit your needs.

For further details about configuring the Scheduling Server, please check this video.

From the remaining list of updates let me highlight the following.

1. Reload results from server in the QA Edition now also gets the failure details (DOM and the image on failure) in case of test failure.

2. Reload Results from Scheduling Server downloads all the results to update them (we were previously skipping those already downloaded locally).

3. Reload from server filters run results by start/date as well as by project.

4. Fixed run results errors after "Revert from source control" operation.

5. We keep the machine on which the test was executed within the RunResult object.

6. We now keep the server/execution machine settings on TestStudio upgrade. Previously when uninstalling TestStudio one was losing the settings and had to reconfigure them.

7. The errors running the Scheduling and Execution Server setup applications due to missing administration rights are fixed.

8. We improved the OS information displayed about the execution machines.

9. We also fixed the missing information about the installed Chrome browser on the execution machines.

10. Fixed incorrect result info for tests failed to execute upon test scheduling.

Last, but not least, do not miss the enhancements of the command line runner. We exposed some arguments so that one can get the run result published on the scheduling server when execution completes. For details, please check this blog post.

For complete list of updates in this release please check the notes here.

You still haven’t had a chance to run the new release? Go grab the bits and give it a go.

We appreciate all the feedback customers share so please keep it coming!

Best Regards,

-Konstantin