Answering What’s New in Test Studio Webinar Q1 2015 Questions

The last Test Studio Release webinar was attended by more than 400 participants, and, as such, we had more than 100 questions asked during the webinar. We addressed most of the questions during the webinar but we ran out of time. So, here's a handful of those unanswered questions, answered.

Q: I'm looking at potentially using Test Studio solution for testing report generation. When checking actual results against expected results, will Test Studio solution highlight the actual differences in the report table, or does it just say there is a difference somewhere in the results?

A: Test Studio solution only tells you that the verification failed. In the test log, it will report what it was expecting to see and what it actually saw. Test Studio solution will also take a screen shot of the application under test at the point of failure. No highlighting takes place.

Q: Do you have translation for DevExpress controls?

A: We do have many translators for Telerik UI controls. You can view the list of available translators by launching Test Studio solution, opening any test project and going to Settings -> Translators. There are three tabs, one for HTML, one for Silverlight and one for WPF. However, you can fully automate DevExpress controls without translators.

Q: Can we capture the screenshots in case of failure?

A: This happens auto-magically within Test Studio solution.

Q: Do we have the capability to define row numbers in the data source, for data-driven testing?

A: A data-driven test will start at the first row and run through the test once for each row of data contained in the data source. You cannot control the order Test Studio solution will use the data, unless you modify the order of the data in the data source prior to running the test.

Q: With the local data User/Password example, is there a way to specify what "row'"of the data you have? For example, if you 10 user names, and you want to use the one in the third row for this login, can you specify row 3?

A: No, this cannot be done with the local data table; however, it can be done with an external data source, such as an Excel file. When you bind your test to the data source ,you have the option of specifying which rows to filter on. 

Q: Can I use Test Studio solution directly test my Web API (i.e. without an webpage)?

A: Because we leverage a first-class coding language in C#, there is a lot of flexibility in what you can customize in code. You can easily include your own libraries and reference them in a coded step.

Q: How does Test Studio solution deal with dynamically generated DOM elements or elements that don't have a specific ID set? Is there a standard way to run tests when "know" ID doesn't exist?

A: Yes, there are default settings you can specify to ignore IDs when crafting find logic.

Q: Can we add elements to an element map without getting to record mode, which is much more efficient when we need to insert new/missed steps in the middle of a script. Otherwise, it's time-consuming to run script and wait to get to an interim step each time, just to add that one missing object.

A: Currently, the recorder is the only way to add elements to the Test Studio element
repository. You can record actions on elements or simply Add Element to Repository. However, we are in the process of implementing new functionality that will allow for this. You can add an element in the Elements explorer without the recorder, optionally adding find logic. When your application is ready for testing, you can connect to your application and associate this element with the real one in the application. We will include this functionality in the next release of Test Studio solution.

 Q: How does Test Studio solution handle storage of a significantly large number of elements, which is commonly required for large automation projects? Are multiple element maps required? How can they be maintained?

A: This is explained well in our KB article on how the Element Repository works.

Q: Regular Expressions usage: Please provide an example of using regular expressions, such as object recognition.

A: There’s an example at the bottom of this online users guide page.

Q: How easy is it to source control the tests?

A: It's built in for TFS and we work very well with any flat file based source control system such as SVN

Q: Can we use Test Studio solution for Telerik WPF?

A: Certainly! Test Studio solution can automate against your WPF application built using Telerik WPF controls–and we even include translators for extended verifications to further boost your productivity.

Q: One thing we have really been wanting to add to our testing suite using Test Studio solution is testing for SQL injection vulnerabilities in our targeted application.  For example, a login using username: test' or 1=1-- is the classic case of SQL injection in a login form.

A: Test Studio fully automates the input of data into the input fields of your application. It will be up to you to figure out how to verify whether or not a security breach has happened. Test Studio solution was designed to perform functional testing of your web application, for instance "did you get the correct response back," not security testing.

Q: Has the IF statement been expanded to do a check against the data, such as If $Address2 is blank, do this else  or do that?

A: This can be done by adding an appropriate verification to check if the field is blank and using that as your If verification logic.

Q: Isthere an API for Test Studio solution? Can Test Studio solution be used to automate data retrieval from a website (scrapping). Does it show the xpath for an element.

A: Test Studio solution  is built upon and uses the Telerik Testing Framework, with its API here.

Q: How do we send a bug as an email to the team?

A: When you set up a Telerik Scheduling Server, you have the option to specify an SMTP server to use. When a test that is part of a test list fails, Test Studio solution can automatically send an email through the configured SMTP server that includes the failure details. Also, using our Execution Extension, customers can implement their own logic. They can create a custom log with information they need about the execution and send them via mail. This is an advanced topic.

Q: Does a QA tester have to have access to the code base in order to use this, or is it all done from the GUI/website?

A: Test Studio solution falls in the category of “black box” testing. It does all of its testing and automation through the UI. It does not need the source code of the application to do its work.

Q: What are the differences between the DEVELOPER and QA versions of Test Studio solution?

A: DEVELOPER and QA versions are old editions of Test Studio solution that no longer exist. Now, when you purchase a license to Test Studio solution, it comes with our standalone IDE (what we used to call the QA edition) as well as our Visual Studio plug-in (what we used to call the Developer edition).

Q: If we purchase perpetual license, can we still useelerik after it expires, or do we have to pay $2,999 every year.

A: With a Perpetual license ,you are free to use the version of Test Studio solution that you currently have installed, forever. However, you need to pay an annual maintenance fee for access to product updates and priority support.  

Keep in mind getting upgrades can be very important. We frequently have to play “catch up” with the browsers as new versions of the browsers come out with new features, or new security policies break our current method of controlling the browser. Thus if you try to stick with a 2-year old version of Test Studio solution, you may put yourself in a position of being unable to run tests on the latest version of browser X. 

Q: How do I automate the dynamic link in the project?

A: Do you mean an element with a dynamicID? If so, there are several approaches you can take. Here is a great blog post I encourage you to look at for guidance: http://telerikhelper.net/2013/12/09/handle-dynamic-id-with-eae-in-test-studio/

Q: Is Test Studio solution good for testing XML controls, or would it be better to use Telerik Kendo UI suite?

A: Test Studio solution automates the UI of an application. That UI is the DOM of a browser, or the Visual Tree of a Silverlight or WPF application. What control is used in the code of the application to generate the DOM or Visual Tree is irrelevant.

 Q: Is Test Studio solution appropriate for testing SharePoint hosted pages and/or code?

A: Absolutely! We have a lot of customers who use this very same combination.

Q: Is it possible to perform an IF on bound data to allow, for example, for different paths for different user roles?

A: No, it is not, and we strongly recommend against such an approach. Each test case you write (and you should be writing out test cases before you even launch Test Studio solution for any serious work) deserves its own unique and separate test. Test Cases should be like “Can a super admin user do X," “Can a limited admin user do X," “Can a restricted user do X” or “Is a restricted user blocked from doing X.” Each of these
should be a separate test case, and a separate test.

The first problem you run into if you try to combine all of these test cases into one test is when the test fails, you don’t know which test case is broken. You now have to spend time picking through your results to determine which test case is broken so you can file a bug on it. This is time that would have been better spent creating another test.

The next problem you run into is test complexity. The more complex your test script is, the more brittle it is, which means more time spent trying to maintain it as the application changes.

By separating your test cases into unique tests, you immediately know which test case is broken, simply by looking at which test failed.

Q: We have a single-page web application that uses nested iframes. We had numerous issues while running tests. Is there any specific method or setting to help us run tests on nested iframes?

A: No, there are no settings or known issues with regard to running a test against an application with nested iframes. I request that you file a support ticket using our online support ticket system, so we can get more details about your specific error and work closely with you to successfully resolve your issue.

Q: How do you perform revision control on your test cases vs. code revision? I have v(x) of code and v(xTest) tests; how can the tester keep track of versions?

A: By itself, Test Studio solution has no revision control. It has only one revision, the “current” revision, contained in the test file and located on disk. To gain revision control, you need to use a third-party source control system, such as TFS, SVN or GitHub, which are all supported by Test Studio solution.

Q: I noticed a "Load" and "Manual" options when creating a test. Can you use this for a Windows Form project?

A: Nope. Load is for Load Testing of a web server. Manual is for Manual testing where the manual steps to perform are written out as steps in the test; the tester follows the instructions contained in the steps and records the results in Test Studio solution as the instructions are manually followed by the tester.