Rich user experiences with HTML5 applications can cause some difficulties for teams trying to get solid user interface (UI) testing around their apps. Interacting with video or audio elements is often troublesome, and the manipulations of a web page’s document object model (DOM) on the fly via asynchronous alterations causes automation frameworks (and automaters!) endless grief in the attempt to get subtle timing issues stabilized.
Telerik’s Test Studio brings several critical features to the game which can help teams create high-value, stable automated test suites that minimize long-term maintenance costs during a project’s lifetime.
Before firing up Test Studio and tearing off on automating every dialog, style, and button of your entire application, make sure you have a good grasp on what you’re targeting for your UI automated tests. UI automation is the hardest of all automated test types, and it’s the most costly to implement. Understanding what in your UI to test, and even more importantly what not to test is crucial for your long-term success. (And sanity, quite frankly.)
Test automation comes in a number of shapes and flavors, and every project benefits from a good mix of automation. A number of authors, including Lisa Crispin and Janet Gregory in their book Agile Testing, have put forth variations testing pyramids showing a mix of various test types. The idea is that the base of any project should be a broad, solid set of unit tests since they’re fast and have few dependencies. The middle of the pyramid consists of integration tests; slower-running tests crossing service boundaries. The top of the pyramid is a much smaller set of carefully chosen UI tests designed to validate vertical slices of your application’s most critical features.
(Interested in more reading on good mixes of automated tests? Go have a read of a review at The Tar Pit.)
“Carefully chosen” is a phrase I use carefully, because it’s important to avoid spending time on automating parts of your system that are better covered with exploratory or other forms of manual testing. Validating layout, styles, and graphics are tasks best left for the Mark I Eyeball. Save your automation work for more important tasks!
Test Studio is Telerik’s functional, load, and performance testing tool. (There’s also a great set of tools for iOS testing. Head over to the Test Studio blog and site for more information on that!)
Test Studio helps you quickly create high-value UI tests that are easy to maintain over the lifetime of your project. Test Studio’s record and playback approach gets you started with a framework of a test which you can tweak and extend as needed for your situation. You’re also able to easily drop to code when you need to do tasks like hitting backing APIs for setup and tear-down operations, or for writing test oracles to validate tests in a database. You can also work within Visual Studio via the Test Studio plugin if that’s where you’re more comfortable.
Recording tests for web applications leverages Test Studio’s translator extensions to open up elements and controls on a page and expose their particular actions and properties for interacting with in tests. Test Studio includes translators for basic HTML elements, and also a great many of Telerik’s custom UI controls.
Each translator provides a unique set of extensions for its specific element or control. Basic HTML translators expose basic properties and validations for elements such as shown below.
Moving up to controls or elements backed by more powerful translators mean you’ll be able to vary the scope of your selection as well as access more actions and validations. The example below shows Test Studio inspecting a RadGrid for ASP.NET WebForms via the Telerik translator for that control.
Let’s use the Kendo Sushi site for our first set of tests. We’ll do a simple addition of a sushi item to the cart. From the Test Studio project, add a new test.
Double-click the test to change to the Test Explorer view, then click the red Record button (1 below). Internet Explorer will launch and attach the recording bar to the IE window (2 below).
Navigate to http://demos.kendoui.com/websushi/#/. You’ll notice the navigation step being added in the actions pane of the Test Explorer window. Click the “Add to cart” button under the Chirachi Sushi offering. You’ll see the corresponding Click of the ListView control added to the actions pane. Now it’s time to get a couple verifications built. Click the hover over highlighting control on recording toolbar—the blue rectangle icon just to the left of item 2 in the graphic above. Now hover over the $21 under Your Shopping. As shown below you’ll see that element highlighted in red, and if you hover there for a moment the blue “nib” will appear. Click on that and you’ll get the Element menu.
Click the lightning icon in the middle to open the Quick Tasks menu. This menu gives you common options that are appropriate for the current control you’re using. The translators define what options are shown here. In this case we see it’s simple to select a validation for the current cart.
Select that verification, then move up to highlight the “1 items” element and pull up the Element Menu. This time we’ll use the Build Verification item to create a different type of validation.
Test Studio’s Sentence Builder enables you to do much more complex verifications on attributes, styles, and other properties. In this case we’re going to select a simple InnerText validation to check the proper count of items in the cart.
The test now looks similar to the figure below.
At this point you may be a bit puzzled because nothing so far has been specific to HTML5. You’re right! Much of web application testing isn’t specific to HTML5—it’s just general HTML testing--but Test Studio is ready when you need to work with something like HTML5’s video players.
In this example, we’ll work with the HTML5 video player at http://videojs.com. We’ll skip the initial steps already shown above and get straight to interacting with the player. Go ahead and start the video playing by clicking the play marker in the center of the video image.
Once the video is playing, highlight the video canvas and pull up the Quick Tasks menu. You’ll note there are video-specific tasks listed here, including the ability to validate the video’s actually streaming.
You can pause and validate that state as well!
Looking for more ways to leverage Test Studio’s ability to interact with Kendo UI controls and well-done sites? Let’s flip back to the Kendo Sushi site and examine two great visual effects, both tied to hover over events.
While I avoid automating style and formatting checks, in this case it may make sense to ensure hovering over particular areas of the application cause appropriate event behaviors to fire off. For example, hovering over the Add to Cart button turns the background red.
Start the same as above: create a test, start recording, and navigate to the Kendo Sushi homepage. Turn on highlighting and hover over the Add to Cart element.
To validate the changing background event requires an additional step we haven’t seen yet: we need to add a mouse hover over. Open the Element Menu and select Mouse Actions.
Select Mouse Hover.
Select Center in the following menu. (This has the hover explicitly occur at the exact center of the element. You can select a specific area if you need.) With the hover event in place you can now use the Sentence Builder to add a Style validation for the background color. Note that you’ll have to manually edit the color as the hovering action will stop when you move your mouse away from the button.
You can follow the same steps to add another Style validation to check the background image properly updates to the right arrow if you desired.
UI test automation is a difficult domain, but Test Studio can help you get great, maintainable tests around your application in short order. Get yourself started with the Test Studio 30-day trial
Jim is an Executive Consultant at Pillar Technology. He is also the owner of Guidepost Systems. He has been in various corners of the IT world since joining the US Air Force in 1982. He’s spent time in LAN/WAN and server management roles in addition to many years helping teams and customers deliver great systems. Jim has worked with organizations ranging from startups to Fortune 100 companies to improve their delivery processes and ship better value to their customers. When not at work you might find Jim in the kitchen with a glass of wine, playing Xbox, hiking with his family, or banished to the garage while trying to practice his guitar.