With its latest R1 2021 release, Test Studio expands the capabilities it offers to testers in an effort to help them modernize their portfolio of testing types and improve test scenario coverage.
Test Studio’s newly launched headless browser execution can significantly improve both delivery speed and release quality. Running tests in headless mode allows you to integrate your UI automation suite into the nightly runs while reducing the risk of test failure. As a result, your team can focus on critical app functionality, eliminate regressions early in the process, and get actionable feedback on release health.
The advantages headless browser testing offers within a modern testing cycle are numerous and yet, as a technique many testers haven't tried yet, it raises a lot of questions to take serious look at.
This blog post will give you a recap of the most interesting and relevant questions that attendees asked during the Test Studio R1 release webinar—carefully selected and answered by Test Studio's automation engineers. In case you missed the webinar, you shouldn't worry. You can watch the webinar recording on demand to catch up on all the top-notch automation tips our speakers Petar and Andy provided during the one-hour session.
For the sake of providing you as extensive an overview as possible in a fairly easy format, I have summarized the questions I am going to cover under the below topics. Clicking on each of them will scroll you to the section you are most interested in reading about.
Let’s jump right into headless testing, which we can say generated by far the biggest interest among the audience.
Be it because it is a rather unknown technique to QAs or an exotic testing type, which has been for quite some time available only to automation engineers with solid coding experience, but headless testing engaged the audience into a quite "geeky" conversation. But being geeky is among the best things about testing.
A decent amount of questions were circling around what headless testing can and cannot do.
Well, headless is the perfect tool if you need faster test runs performing very frequently (even a few times per day). When the release approaches, you can run the tests against the fully rendered UI of the browsers you need to test your app against.
Anything feature-specific that you need to check against a browser UI or a browser version, for example verifying login functionality of your application on the latest Chrome version, requires running a traditional web test using the browser or browser version in question.
As far as functional testing is concerned, Test Studio supports it with both "headed-browser" and headless testing, allowing you to verify both the UI and the functionality of your application. It depends on the specific use case which one you should use. Our friends from Abstracta cover the case of functional testing quite exhaustively in their Beginner's Guide to Functional Test Automation.
Another ideal scenario for headless testing is test scheduling and remote test execution. Using the Test Studio Run-Time component, you can schedule your runs during the night and run your tests in a headless fashion to both speed up execution (up to three times) and make sure your tests will remain stable.
Several attendees were concerned about how to overcome the limitation of not being able to perform other tasks while a test is running on their computer.
Headless testing with Test Studio allows testers to use their machines while tests are running. You can, for example, check/inspect other UI tests on that machine or do your daily routines. The only thing that we recommend you avoid is recording new tests while the headless ones are running.
Another attendee asked:
To use headless, do I need to create two separate tests or run the same one in two different modes—by rendering a browser's UI and headlessly?
Headless browser execution in Test Studio allows you to run the same test—a newly created or an existing one—in two different modes without any additional adjustments.
Will headless browsers have any impact on element identification techniques was another question that unveils how the new features integrate into existing testing architectures.
Test Studio uses a hybrid element find logic utilizing both element ID and image, which in general makes tests more stable and less flaky. This logic is fully integrated with headless browser execution, which in fact is expected to improve test stability significantly as it doesn't require rendering an actual browser’s UI.
Another user wondered if it is possible to have desktop command in a script while executing tests headlessly. Test Studio not only allows you to have desktop commands but will take care of them, making sure these are properly executed.
Here are a few more questions whose answers seem to be quite useful for the tester.
Will Test Studio consider actions that will only succeed when triggered by trusted browser events?
Test Studio’s special real user action steps trigger mouse and keyboard events as trusted browser events which also works in headless mode.
Can you set breakpoints in headless mode?
You can set breakpoints but you will not see much since there will be no GUI. Test Studio’s visual debugger will be working though so you can get, for example, the DOM explorer through that.
When executing a headless test, is it possible to increase the risk of false positive results?
That really depends on how your tests are configured. However, in general headless execution should provide the same success ratio as non-headless.
Test Studio’s R1 2021 Release introduced another useful feature that users have been wishing for. Automated testing of web applications is now extended with built-in PDF content validation features to support all types of PDF-dependent websites, internal applications (CRM, BI or ERP systems), and Robotic Process Automation (RPA) workflows.
Expectedly, PDF validation generated lively discussion, as PDFs are rarely supported by test automation tools through out-of-the-box features. Most automation tools on the market require installing add-on’s or third-party tools to cover specific PDF-related cases, which more often than not turns PDF automation into quite a cumbersome task. With Test Studio though, as long as you have the browser extension for test recording installed, you can validate PDF content inside the Test Recorder without workarounds or additional hassle.
While preparing the webinar, we assumed that most users will be interested in applying the PDF validation features to internal applications. What we observed, though, is an emerging need for such capabilities to automate customer-facing user interfaces. This is mainly due to trade becoming contactless during the raging pandemic, which forced a lot of businesses to go online (Forbes).
We had an attendee asking whether it is possible to verify a PDF that is generated in a frame within your web application UI—for example a financial report that gets displayed to the end user within the web UI.
Here's the answer. Test Studio’s OCR features are explicitly designed to help solve such scenarios. OCR (Optical Character Recognition) allows the user to validate text and images on web alike in PDFs that are dynamically generated and represented in the web UI.
Another important aspect is how PDF validation handles elements. One of the attendees asked a very relevant question related to how the elements are identified within the PDF.
The beauty of the PDF features is that all validation actions you are used to from validating web elements apply to PDFs. Hence, PDF elements are added to the Element Repository and are handled by Test Studio just like normal web elements. The Element Repository is shared across test suites and projects and helps you eliminate duplication of work while creating and maintaining tests.
Another important fact for Telerik customers is that PDF content validation can be a very useful tool for automation of Telerik Reporting-built applications.
A significant number of attendees identified themselves as developers involved in testing, which for the most part didn’t surprise us—especially when it comes to agile environments, where testers and developers work side by side.
Bringing UI automation into CI/CD (continuous integration/continuous delivery) is no longer just a best practice, but a popular approach that has proven to bring efficiencies to delivery teams. Test Studio, in fact, comes with both a standalone IDE and dev edition in form of a plugin that allows developers to automate tests in Visual Studio directly. That is an essential aspect for agile teams when it comes to team collaboration and establishing a testing architecture centered around release health.
Can QAs create tests in the standalone Test Studio so that developers can use/run these in the Dev Edition came as a question from the audience.
No matter where the tests or test lists are created, both editions are fully integrated and allow tests to be accessed and run within both. This not only fosters collaboration but provides immense flexibility to mixed teams.
Another webinar attendee, let’s call him Nick (anonymized for the purpose of presenting this case), shared his team’s bittersweet experience with how they test their ASP.NET MVC–based application. Nick’s team, which consists mainly of developers, relies mostly on backend testing around the builds without applying any sort of UI testing. Nick admitted that this is causing his team a lot of hassle since a human must manually work through all scripts.
Well-developed testing architectures created around the needs of both the team and the application under test and mindfully integrated into the delivery pipeline could save a lot of time while improving both release and application quality.
A few of our listeners apparently knew that already because we received quite an impressive amount of questions around the integration of Test Studio with various source control platforms—Git, Jenkins, Azure DevOps.
But most importantly, some of the questions were directly pointed toward test scheduling and remote execution, which speaks of the changing culture among testers to apply strategic approaches of integrating testing into the delivery pipeline and increasing the value of testing as part of the release cycle.
Another topic that generated decent interest was testing microservices. The more complicated and demanding the software architecture is, the more critical it is to do proper testing. Microservices running in the cloud, including cloud providers such as AWS, KAFKA Kubernetes, etc., ensure software architecture flexibility but could be a real deal-breaker for testing tools. As long as a microservice has a web UI that can be accessed from the testing machine, it shouldn’t be a problem for Test Studio to handle testing this microservice.
We hope that we provided useful answers to these burning questions. Should we have left any question around the release unanswered, feel free to contact us or leave your question in the comments section below.
The Test Studio R1 2021 release ships with features that support both less experienced and senior testers, enabling them to do a better job and contribute to the organization’s delivery process through optimized testing workflows.
Asya Ivanova is the Product Marketing Manager for Telerik Test Studio. A passionate technology enthusiast, she has product experience within BIM, Master Data Management and pro audio. Together with the Test Studio team she keeps an eye on ways to make test automation easier for the QA engineer. Asya is a hi fi geek, spending her time with some good old and new album records. Connect on LinkedIn.
Subscribe to be the first to get our expert-written articles and tutorials for developers!