What is Headless Browser Testing, When (and Why) to Use It

Headless browser testing improves both the effectiveness and efficiency of your testing process while integrating quality assurance with software delivery.

In the end, the only tests that matter are end-to-end UI-driven tests. These are the tests that begin and end with the application’s interfaces and prove that the application—with all of its events, queues and microservices—allows the stakeholders to meet all of their goals.

But there are some inherent problems with UI-driven testing. Stability is one: Even when the application is working correctly, UI-driven tests occasionally fail when interacting with the browser. Performance is the other issue: UI-driven testing is sloooooooowwww compared to other kinds of automated testing.

What Is Headless Browser Testing?

There’s a solution: Headless browser testing. Headless browser automation uses a web browser for end-to-end tests but skips loading the browsers’ UI. That means the HTML page being tested isn’t rendered (so everything runs faster) and your tests bypass interacting with the page to manipulate the browser more directly (eliminating failures due to UI-related interactions). Your end-to-end tests are both more efficient and more reliable. If performance or stability are the reasons you’ve been avoiding end-to-end UI-based testing, headless browser automation may make it possible for you to add end-to-end tests to your testing process.

But headless browser testing shouldn’t be thought of as just a replacement for UI-based testing. Headless browser testing creates new opportunities by letting you adopt “shift-left” design thinking and move toward integrating Quality Assurance (QA) into your software delivery process (what’s referred to as “QAOps”). The net effect is that “headless browser testing” not only makes your testing more efficient, it also makes your testing more effective.

And, by the way, “headless browser testing” is a mouthful. Let’s just use “headless testing.”

The Opportunities

The stability and performance problems associated with end-to-end, UI-based testing means that end-to-end tests are usually done less frequently than other tests—often only at the end of the development process, just before release as part of the QA process. This has, at least, two unfortunate results: Any problems unearthed by end-to-end testing are discovered very late in the testing process, and a division is created between the software delivery and QA processes.

Here’s where headless testing steps in. By not invoking the browser’s rendering engine, a headless browser test can run anywhere from two to ten times faster than the equivalent UI-driven test. This speed improvement allows you to shift end-to-end testing left, moving it out of pre-release testing and into nightly test runs—your testing becomes more effective by letting developers identify and address problems earlier. If you’re already doing end-to-end testing as part of your nightly testing, then you have a chance to free up resources currently soaked up by UI-driven testing—your testing becomes more efficient.

While faster is better, that move is only possible because headless testing, by bypassing the browser’s UI, also improves the test’s stability. That stability matters: While pre-release testing is infrequent enough to allow for some instability, it’s an axiom that nothing (repeat: nothing) can disrupt the nightly build.

Moving end-to-end testing to your nightly runs means that what was once part of the pre-release QA process has now been integrated into the software delivery process. The result is to reduce the division between the two teams and start creating a QAOps mentality.

The Costs

There are some conversion costs, though. One of the benefits of UI-driven testing is user scripting: Users can generate tests that are meaningful to them by interacting with the application to create test scripts (including validating results) without necessarily relying on testing frameworks. Some developer support is still required, though, because it’s an unusual environment where end user scripting handles all of the required tests.

More good news: Headless browser testing can use your end-user generated scripts. However, any developer-generated tests that are part of your end-to-end testing will have to be rewritten to function in the “headless” environment. Developers writing tests for the headless environment will need to develop some new skills.

It would also be a mistake to think that headless testing will let you drop UI-driven testing. As modernization strategist Jim Holmes recognized back in 2013, “Starting a browser session does indeed slow down tests … but [headless tests] do not replace browser-based functional tests. The whole point of functional UI testing is to ensure your application works in the various browsers with all their unique quirks and ‘personalities.’ Headless tests are a great, fast integration-level test, but you shouldn’t skip full-blown browser testing.”

Much has happened since then, including the adoption of Chromium as the base for many browsers, but Jim’s point still stands. In fact, products like Telerik Test Studio are still making this message a priority—both UI-driven testing and headless browser testing matter and should go hand-in-hand—and Test Studio will support both tests with solid capabilities for both technical and less technical users as of the R1 2021 release.

Since both UI-driven testing and headless browser testing are end-to-end testing, there will almost certainly be some overlap in tests between the two sets of tests. By itself, that’s not necessarily a bad thing—the biggest cost being redundant testing when both sets of tests are run together. If that inefficiency is bothering you, it can be addressed by skipping headless testing during the pre-release process (after all, the code passed your headless tests last night). This does add some complexity to test management if you’re currently using the same process for both your nightly and pre-release testing.

Headless browser testing provides efficiencies that allow you to both apply shift-left thinking and start to implement QAOps. By moving end-to-end testing to earlier in your development process, you make the tests more useful, an obviously a good thing.

But, to use a phrase that’s been hanging around since the 1930s, “There ain’t no such thing as a free lunch”: You’ll need to commit developer resources into building the necessary non-scripted tests and commit test management resources to crafting the right processes for your organization. Fortunately, integrating QA with software development through QAOps can help provide the necessary resources.