Telerik blogs
Test StudioT2 Light_1200x303

Here’s how to convert any test into a data-driven test that lets you prove your app works with every set of data that matters.

So you’ve created your first end-to-end (E2E) test and it’s proving that, even as more changes are made to your application, your application continues to process your test transaction “as expected.”

Test Studio showing the results of running a successful test

And you’d like to run that test again, but with different data. You’ve done your equivalence partitioning to determine what groups of data exist that are relevant to your application (i.e., where “significant” differences exist in the application’s potential inputs). You’ve also pulled at least one set of data from each group. Now you want to prove that your application works “as expected” with each set of data that you’ve identified.

What you don’t want to do is write a new test for each set of data. While the data is different, the application should be able to process each set of data just like the inputs from your original test. You also don’t want to make your test so complicated that the test itself needs to be tested. One of the benefits of Test Studio’s data-driven tests is that you can incorporate multiple sets of data into a single test without violating that rule.

For example, my case study is built around updating department data in the fictional Contoso University app. (Feel free to download both my version of the Contoso University application and the Test Studio project I used to test it.) The user should be able to change the department’s name, its start date, budget and administrator. Within the budget alone, I want to make sure the application will accept “typical” budget amounts (e.g., $5,000) and very large numbers (e.g., $1,000,000). The application should even accept negative numbers like -$200 (some departments actually bring in more money than they spend and, as a result, return money to the university).

The test script for all of these sets of data looks the same—just the inputs change (if the test script is different from one set of data to another, it suggests that you’re testing a different transaction). Eventually, you’ll want to check that the application handles bad data correctly, but I’ll cover that in a separate post.

There are real benefits to creating one test that handles all the data items for this transaction. If, for example, you do have to come back and change your test, you’ll only have to update one script instead of many similar scripts. It may make even more sense to create a data-driven test when you have only a single set of data. By moving your test values out of your test and into a separate source, it’s not only easier to change that data (as you’ll see here), but you can share that dataset among multiple test scripts, when that makes sense.

It’s a good idea to start off by creating a test without worrying about making it data-driven (much like the test I created in the post at the start of this article). Test Studio makes integrating multiple sets of data into a test easy.

Setting Up Your Data

Once you’ve created a standard test, your next step is to create the sets of data you intend to use with your test—your data source. You don’t need to do a lot of planning here because, as you’ll see, it’s easy to go back and modify your data source if you need to make a change.

Test Studio provides a convenient built-in tool for creating and holding your data (you can find it on the “Local Data” tab at the bottom of your test steps). However, you can use almost any tool that holds data to drive your test (including pulling data from an actual database). For this case study, I’ll use Excel because, I assume, you’re already familiar with it.

The list of test steps with the ‘Local data’ tab at the bottom of the steps circled.

After you start Excel, you must first set up the columns that will hold the data you need. You can do that by scanning your test steps and inventing some appropriate names for the data the columns will hold. Looking at both the Edit Department page and my sample test, I’ll need one column for each field the Department Edit page lets me update. I’ll call three of those columns DeptName, DeptBudget and DeptAdministrator.

The Contoso update department page showing three textboxes (Name, Budget, and Start Date) and a dropdown list labeled Administrator, with a Submit button at the bottom of the page. At the top of the page is a checkbox labeled ‘Create Bug’

I’ve used a coded step to handle entering the date and, because of the way I’ve written that step, I’ll need three more columns for it: DeptStartYear, DeptStartMonth and DeptStartDay. In addition, for the purposes of this case study, I added a checkbox to the page that lets me create a failing test whenever I need to demonstrate an error in this case study. I’ll add a column called InsertBug for that.

Finally, I’ll put in a column called Notes so that I can document the intent of each test row.

I don’t need to set up columns for anything that doesn’t change from one test to another (e.g., clicking the submit button at the end) and, as my Notes column suggests, I don’t have to use every column I set up in my data source. Purely for documentation purposes, I give my worksheet a name (DeptEditTests), save it and close it.

The result, with my test data, looks something like this:

An Excel worksheet name DeptEditTests with the columns described in the article. There are three rows with identical data in each column except for two: The DeptBudget column has different values in each of the three rows (1000000, 5000, and -200) and the Notes column also has different values in each of its three rows (“Standard Budget,” “Big Budget,” and “Negative Budget”) that describe the purpose of each row.

One word of warning when working with Excel: Make sure that the rows that follow your test data really are empty. A blank space in a cell in a row following your test data will convince Test Studio that it should feed that row to your application as part of your test. Select the row following your test data and press the Delete key to clear it.

Now that I’ve prepared my data, I can add my Excel spreadsheet as a data source in Test Studio. Back in Test Studio, I first switch to the Project tab and then, in the Data Sources section, click the arrow under the Add+ icon to get a list of the formats that Test Studio accepts. I select Excel File, which opens a file open dialog. In the dialog, I surf to the folder with my spreadsheet, select it and click the Create button. Test Studio copies the file to my test project’s data folder.

The Test Studio menu bar with the Project tab selected and the dropdown list under the add icon displayed showing the four options: Excel File, CSV File, XML File, and Database. To the right of the Add icon is the Bind Test icon.

With my data source loaded, I can bind it to my test by clicking the Bind Test icon. This opens another dialog with a Select Data Source dropdown list at the top. Clicking that reveals my Excel data source, which I select. That causes the dialog to display another dropdown list, labeled Select Table—clicking that lets me select which sheet in my Excel spreadsheet I want to use for this test script. When I select my spreadsheet, the dialog gives me a preview of my data.

The Bind test to a data source dialog showing the dropdown lists with the DepartmentInfo Excel spreadsheet in the top dropdown list and Sheet1 selected in the bottom list. The four rows of the Excel spreadsheet (the heading row and the three rows of data) are displayed below. At the bottom of the dialog is a Bind button.

Clicking the Bind button at the bottom of the dialog completes adding the spreadsheet to my project. I’m ready to have my test script use this data.

Binding a Test Step

All that’s required to data-bind a test step is to tie a column in your data source to the value used by the test step. You do that in the Properties window for the step you want to data-bind. After you select the step, you’ll find the Properties window to the right of your step list, tabbed together with the Step Builder—just click on the Properties tab to show the window. I’ll start by binding the step where my original, non-data-driven test currently types in the number 200.

The list of steps in a recorded test with the “Enter text ‘200’ in ‘Budget Text’” step expanded to show that its Text value is currently set to 200. The Step Builder pane shown to the right of the test script. At the bottom of the pane the tab labelled “Properties” is circled.

Once the step’s Properties tab is displayed, you’ll find the Bindings builder button (the button with the three dots) at the top of the property pane. Clicking the Bindings button will let you replace your fixed value with the name of a column from the data source bound to your test.

The step’s Properties tab with the top line – labelled “Bindings” – highlighted. The line has a button with three dots at the right hand end.

When I click the builder button, I get a popup dialog with a dropdown list that displays the column names from my Excel spreadsheet: DeptName, DeptBudget, DeptStartMonth and so on. Since this step is entering the budget data, I select DeptBudget and click the Set button on the dialog. The title of my step in the test script is updated to show that the step is now Data Driven. I then repeat that process with the other data entry steps in my test script.

The dropdown list that’s displayed after clicking the Bindings builder button, showing the columns from the Excel spreadsheet.

When I bind the dropdown list, however, the Bindings dialog gives me a different set of options: I can set one or more of SelectionText, SelectionValue and SelectionIndex. I entered the name of my administrator into my spreadsheet and, on the Department Edit page, the administrator’s name is displayed in a dropdown list as the list’s Text. As a result, I’ll need to set the SelectionText option to my Excel spreadsheet column to have the dropdown list select the correct option using the data from my data source.

The Bindings dialog for a dropdown list showing three dropdown lists labeled SelectionText, SelectionValue, and SelectionIndex. The first dropdown list – SelectionText – has been set to DeptAdministrator

In addition to binding my actions, I also need to bind any verification steps I set up when I created my manual so that I’m checking my results against what’s being pulled from my data source (fortunately, binding a verification step is just like binding a textbox). This is typical of data-driven tests: You have one set of columns for your inputs (what you think of as your “test data”) and another set of columns holding the data you’ll use to validate the test (your “expected results”).

By the way, if your application includes checkboxes or radio buttons and you’re binding data from an Excel spreadsheet, just use Excel’s TRUE and FALSE values in your spreadsheet columns when setting, unsetting or verifying them.

Updating the Data Source

As I’m binding my verification step, I realize that I could use two columns to represent my budget amount. When I enter the data on the Department Edit page, I just want to enter digits (200, for example) because that’s all the textbox will accept. But, because of the way that the data is displayed on Department Display page, when I want to verify the data, I want to use the character string “$200.00”.

My verification step can handle that difference if I use the “Contains” operator in my verification test (the string “$200.00” contains the digits 200, after all). But I’d be more comfortable using the “Exact” match to make sure the data is being displayed correctly (to ensure the negative sign is in the proper place, for example).

Fortunately, updating my data source is easy: First, I click on the Manage icon in the Data Sources box on the menu to display a list of my data sources.

The Data Sources dialog showing the project’s single data source as the full path to the Excel spreadsheet in the data folder of my Test Studio project. To the right of the spreadsheet path is a pencil icon and a trashcan icon.

Clicking on the pencil icon for the Excel spreadsheet driving this test will open my spreadsheet in Excel. I add a new column to my spreadsheet called DeptBudgetChecked and fill it with the formatted display I’ll get on the Department Display page (make sure you enter the string with an apostrophe in front of it—‘$5,000.00—or Excel will store it as a number). I then just close and save my spreadsheet to update my data source in Test Studio.

An Excel spreadsheet holding the row of headings and data from the earlier example with a new column added. The new column is headed DeptBudgetChecked and contains a formatted version of the data in DeptBudget column.

Back in Test Studio, in my verification step, when I return to the Bindings dialog, I find the DeptBudgetChecked column is there for me to use. I can use the same process to add, update or remove the data that drives my test.

Running the Test

After I’ve bound all my steps, I can run my test. With my sample data, I’ll see the test run three times as Test Studio processes each row in my dataset. If all my tests pass, a green bar will be displayed at the top of my test script.

The test script after a successful run. At the top of the test script a green bar is displayed with a dropdown list in the middle. The dropdown list shows “Iteration #1: (DeptName = English” and then is cut off by the width of dropdown list..

If one of my test runs has failed, I’ll get a red bar at the top of my test script.

The test script after a failed run. At the top of the test script a red bar is displayed with a dropdown list in the middle. The dropdown list shows “Iteration #1: (DeptName = English” and then is cut off by the width of dropdown list. At the bottom of the test script is a pane labelled Step Failure Details with tools for recording and analyzing the failing test.

To determine which run(s) failed, you can use the dropdown list to select each run. As you select each run, those runs which failed will have a Step Failure Details pane beneath your test script.

The test script after a failed run with the dropdown list in the middle of the red bar at the top expanded. The dropdown list shows three entries beginning “Iteration #1”, “Iteration #2”, and “Iteration #3” along with the data used in each iteration. The dropdown list’s options are displayed across the width of the Test Studio screen

The runs that succeeded will have a Step Result Details pane beneath your test script.

The test script after a failed run will also show steps that have succeeded, with green checkmarks

With data-driven testing, effectively every row in your data source is another test. With a single test script and a data source with multiple rows of data, you can start running more tests earlier (and cheaper) than if you had to craft individual test scripts.

Maintenance becomes easier and faster. Also, as you come up with more test cases, you just add them to your data source; if your application changes and starts to produce different output, you just update your “expected results” columns. Either way, you don’t have to write new tests or make changes to an existing test. It’s hard to beat easier, cheaper and faster, but data-driven tests can let you do all three.


Learn more about Test Studio, and start your free trial today!


Peter Vogel
About the Author

Peter Vogel

Peter Vogel is a system architect and principal in PH&V Information Services. PH&V provides full-stack consulting from UX design through object modeling to database design. Peter also writes courses and teaches for Learning Tree International.

Related Posts

Comments

Comments are disabled in preview mode.