The Hidden Costs of Using Selenium for UI Test Automation

Many teams use Selenium UI tests because they are free and open-source. But Selenium comes with some hidden costs you should consider before selecting your automated test tool.

There are so many choices at an impatient user’s fingertips today. It’s essential to design a fast, responsive and intuitive user interface. Even with a well-architected set of APIs, it can mean little if your UI provides a poor experience. For example, a Google study has shown that 53% of visits are abandoned after 3 seconds of load time, and 79% of people are more likely to return if a site is easy to use.

With this in mind, a UI testing strategy is a must. Manual browser testing is a good first step. To keep your QA team (or yourself) sane, though, you’ll need an automated UI testing solution. This allows you to automate painful testing tasks at scale, and there certainly are a lot of choices in the open-source community, and in the marketplace, today.

When moving to automated functional UI testing, many teams first lean toward free open-source tools such as Selenium.

What Is Selenium?

Selenium is a free open-source automated testing framework used to test web applications across many different platforms and browsers. It provides a single interface to automate browsers.

The most popular Selenium tools involve Selenium IDE, which allows you to record, edit and debug functional tests, and Selenium WebDriver, an automated framework for testing across multiple browsers. It provides drivers for C#, JavaScript, Python, Ruby and Java.

Is Selenium Truly Free?

While Selenium is budget-friendly and doesn’t cost any money, is it really free? Here are a few limitations that might lead to a high initial cost, especially when it comes to maintainability.

• Recording and playback. For recording and playback capabilities, you’ll need to use Selenium IDE, which is only supported on Chrome and Firefox. You’ll need to execute tests on each individual browser.
• Steep learning curve. Most teams will lean on Selenium WebDriver, a 100% coded solution. It’s extremely powerful but requires a lot of experience and coding to wire up your ideal testing solution. As development teams are encouraged to own their apps from end-to-end—including infrastructure and testing—will you need your test infrastructure to be built and maintained only by people with coding experience? In these cases, it can feel like you’re caught in the middle: Selenium IDE is limiting but more accessible to all, yet Selenium WebDriver is powerful but requires different skill sets.
• Requires experienced engineers, who are expensive and in high demand. To leverage CodeDriver, it requires coding experience, typically from software engineers or SDETs (Software Development Engineers in Test). These engineers are in demand and can often be difficult to retain. What happens if they leave? It’s often hard to replace these skills easily and replacing them with engineers with less automation experience can lead to a Selenium solution that’s hard to maintain. In many cases, it causes organizations to spend a lot of money and time to hire a new expert. Alternately, easy-to-use commercial tools can be utilized by all team members.
• Maintaining elements in Selenium IDE can be a chore. Selenium tests rely on a single element identifier. In the case of the Selenium IDE, it stores in each step of the test. When you change identifiers, you need to change them everywhere. This can be mitigated by using the page object pattern, but it showcases how your code needs to rely on solid design principles to avoid wiring up spaghetti code.
• Fragmented technical support model. While Selenium has strong community support, there’s not a streamlined way to get support. If you don’t get the community support you need, you can’t email or call a devoted support team.

While you don’t have to pay a license fee for Selenium, you should be aware of the Total Cost of Ownership (TCO) before you Take Care of Business (TCB).

Exploring Telerik Test Studio

While commercial tools come with a price tag, it can be well worth it when you think about Selenium’s total cost of ownership, especially when it comes to test maintenance costs. If you take a look at Telerik Test Studio, you’ll find a single end-to-end automated testing solution that can be used by everyone on the development team.

Consider the following advantages in using Test Studio for your UI automation testing needs:

• No element duplication. Test Studio stores element locators in its element repository, meaning you don’t have to worry about duplication of element identifiers. When your UI changes, go to the element repository and update the elements in question, and your updates will be reflected in all the tests that use it.
• You can reuse tests out of the box. With Selenium UI tests, reusing tests and components can be tricky. Selenium IDE doesn’t allow you to reuse tests as blocks, and doing it in WebDriver requires a skilled development team. Test Studio allows you to reuse tests to avoid unneeded duplication.
• You can use Test Studio as a standalone tool for both developers and QA. With Test Studio, there’s not a gap between a test recorder like Selenium IDE and a coded solution like WebDriver. With Test Studio, testers can design tests through its recording capabilities, then hand them over to developers for complex scenarios. In addition, Test Studio also ships with a Visual Studio plugin that developers can use.
• No need to install a lot of required third-party dependencies. With Selenium UI tests, you’ll need to find your own build server, wrap your WebDriver code in another testing framework, and deal with a lot of setup and coordination. Even more, Selenium doesn’t offer any reporting features. Test Studio ships with a scheduling server and allows each testers to schedule their test runs, saving a lot on administration. Test Studio also has robust reporting capabilities out of the box.
• Test Studio adapts to technology changes. Many open-source testing frameworks don’t have Day-Zero support on substantial changes, especially when it comes to new browser versions. You can depend on Telerik Test Studio to provide Day-Zero support on many of these changes. As an example, many testing frameworks have struggled with locating new Angular attributes, while Test Studio had Day-Zero support. In these cases, users only had to update Test Studio’s find logic to include and prioritize these attributes.
• Dedicated support. With Test Studio, you have a dedicated support team to help you with any issues. You can depend on Telerik for in-depth expertise and fast turnaround times.

Wrap-up

Let’s recap the pros and cons of using Selenium.

Pros of using Selenium:
Selenium is a wonderful, flexible and powerful tool for automating UI tests. It’s battle-tested and is a free, open-source tool with solid community backing and support.

Cons of using Selenium:
However, it also comes with additional costs in terms of maintenance and time.

If you’re looking for an end-to-end solution with centralized configuration and management, reporting and straightforward setup, sign up for a trial of Telerik Test Studio today.