We are setting up our automated test structure. We have found that we will be using test as step quite extensively. With our current structure all tests for a particular function are in one area, i.e. login. Consequently we have on Test and all of the tests it uses in one folder. This will quickly lead to lots of confusion, which tests call the others and which are called, as the number of tests grow.
I thought of having all the master tests at one project level with the tests as step as a sub-project. This won't make it obvious which tests call which test as step.
Is there a better way or some feature I'm missing that will solve this problem better?
Thanks tor your help,
Gregg
I thought of having all the master tests at one project level with the tests as step as a sub-project. This won't make it obvious which tests call which test as step.
Is there a better way or some feature I'm missing that will solve this problem better?
Thanks tor your help,
Gregg
10 Answers, 1 is accepted
0
Hello Gregg,
Managing your tests on different folder levels will be your best option. We have an existing feature request to add a centralized place to know what tests are being used by other tests. You can go to our feedback portal to vote for its inclusion in future versions of Test Studio.
Regards,
Rodney
Telerik
Managing your tests on different folder levels will be your best option. We have an existing feature request to add a centralized place to know what tests are being used by other tests. You can go to our feedback portal to vote for its inclusion in future versions of Test Studio.
Regards,
Rodney
Telerik
0
Bob
Top achievements
Rank 1
Rank 1
answered on 28 Aug 2013, 06:18 PM
Can you lower the number of tests in your suite to help alleviate this problem?
For example, if you're using a separate test to login with different usernames and passwords, you could data drive this instead, making all logins run from the same test.
Typical Login Script:
Navigate to Login Page
Enter Username - Data Drive This!
Enter Password - Data Drive This!
Submit
You could then use local data, spreadsheet data, or code to drive those steps. It also adds the flexibility of allowing you to run the same test with multiple usernames and adjust passwords through find and replace (in excel), especially in systems where a password change is required every so often.
For example, if you're using a separate test to login with different usernames and passwords, you could data drive this instead, making all logins run from the same test.
Typical Login Script:
Navigate to Login Page
Enter Username - Data Drive This!
Enter Password - Data Drive This!
Submit
You could then use local data, spreadsheet data, or code to drive those steps. It also adds the flexibility of allowing you to run the same test with multiple usernames and adjust passwords through find and replace (in excel), especially in systems where a password change is required every so often.
0
Hi,
Boyan Boev
Telerik
@Robert, thank you for your input and helping others. I have updated your Telerik Points.
@Gregg, if you need any additional assistance, please do not hesitate to contact us.
Boyan Boev
Telerik
0
Gregg
Top achievements
Rank 1
Rank 1
answered on 03 Sep 2013, 01:57 PM
wertwert
0
Gregg
Top achievements
Rank 1
Rank 1
answered on 03 Sep 2013, 02:17 PM
Since I can't see what I'm typing, it seems to type in white for this site, I'm attaching my reply as a file.
Why does Telerik do things likke this that make simple things so hard!@?
0
Hello Gregg,
Ivaylo
Telerik
Unfortunately the archive file attached is empty. Could you please add the appropriate file in the archive and reattach it.
Looking forward to hearing from you.
Ivaylo
Telerik
0
Gregg
Top achievements
Rank 1
Rank 1
answered on 06 Sep 2013, 06:18 PM
This should have the file in it.
0
Bob
Top achievements
Rank 1
Rank 1
answered on 06 Sep 2013, 08:20 PM
For the sake of anyone else who might need help with this, here's Gregg's message which he attached to his previous post.
//////// Q('.'Q) Greg's Message Q('.'Q) /////////
Hi Robert,
We are data driving as much as we can. That's actually the source of the dilemma.
Here's a typical scenario. Set up a "driving" test, we've found this works best for us. That test calls a login test, Step A test, step b test, test C test step D test then log off test.
Except for login test none of these tests can be run without some other test in front of then to leave you at a particular screen. Leaving all these tests in one directory will lead to a terrible mix of tests that can be run alone and ones that can't. A totally unworkable mess once we get more than a few tests going. So, the driving tests need to be in one place and the others in some other place. Ideally we would know which driving test calls which driven test, either based on organization or a feature that will tell us which tests are called by the given test.
Thanks for your suggestion.
Gregg.
/////////// Q('.'Q) End Greg's Message Q('.'Q) /////////////
Gregg,
I always have my steps organized in separate directories from my tests. Their sub directories mirror the actual tests so that it's easy to find which test goes with what, or they reflect in what piece of the puzzle they fit.
Example Scenario:
Bob wants to write a test to create an item for his hipster shopping site on the admin site, then check to see if that item appears. There are many different ways to configure an item, and certain combinations create different results. He has 4 pages to navigate to in order to setup the item.
Page A: Properties - Defines the name of the item, it's unique item code, and the type of item. After this, you click "next"
Page B: Inventory - Defines the number of items in stock, and the minimum and maximum a person can order at once. After this, click "next"
Page C: Pricing - Define the price of your item, the shipping cost or, optionally, check a box that says the item is FREE with another purchase. After this, click "next"
Page D: Category - Define which category the item will be displayed in. After this, click "create and preview"
There are many ways Bob wants to test these pages. First, an item with the type "unappealing shirt" can not be marked "FREE". Second, the maximum number of "classy hats" is always 10. And finally, items that are FREE are always automatically added to the "SALE" category in addition to whatever the user defines.
Bob realizes that some parts of his test are reusable. Since his environment isn't fresh each time, he navigates to an online GUID generator using his library of universally useful testing scripts to generate a unique ID for his item. Next, he uses a step required in most tests for his site - a login test to his admin portal in order to add items. These tests are named "GUID" and "LOGIN".
Bob then creates a data driven test to add in the name and unique ID from his extracted GUID from his earlier test, then select an item type from the drop down menu using a data driven element which he reads from his spreadsheet in column "ItemType". This test is named "Page A".
Next, he creates a two tests for Page B. One test enters all the values (maximum, minimum, inventory) from his spreadsheet. He calls this test "Page B Default". His second test only adds data for the minimum and inventory from the spreadsheet. This test checks to make sure that the maximum was automatically set to "10" as defined by the requirements above. He calls this test "Page B Classy Hat".
He creates three tests for Page C. His "Page C Default" test data drives the price, ship cost and ignore the "Free" column. His second test "Page C Free" does what the default test does, but also checks the "Free" box which brings up a dialogue box requiring him to select what items the person needs to purchase to get that item free. Finally, he has "Page C Unappealing Shirt" test, which ensures that the "FREE" checkbox is disabled.
His two tests for page D either select a category from his data (Page D Default) or ensure that "SALE" is selected, as is required for anything "FREE" (Page D Sale).
Phew! That's a lot of tests! Mixing and matching these will provide Bob flexibility in running his tests. He may have one go through all the defaults, one with "FREE" selected, but otherwise all defaults, etc. Problem is, none of these tests run properly by themselves as they're all pieces of a series of tests which will make sure that each permutation of the item creation works.
Knowing that Test Studio Test Lists are formed from a screen that organizes them top to bottom, he opts to place the steps in directories as follows. Tests are in BOLD. Directories are underlined
1 Admin Tests
- Item Creation < This is where the "macro" level tests are stored
-- Default Item Test
-- Classy Hat Test 1
-- Classy Hat Test 2
-- Unappealing Shirt Test 2
2 Admin Test Steps
- Item Creation Tests < This is where the pieces are stored. Note: the directory has the same name as the tests that call these
-- Page A
--- Page A Default
-- Page B
--- Page B Default
--- Page B Classy Hat
-- Page C
--- Page C Default
--- Page C Sale
--- Page C Unappealing Shirt
-- Page D
--- Page D Default
--- Page D Sale
- Common Tests < This is where tests that could run in any test
-- Login
3 Library < Common tests for all his projects are stored here. (Tip: Use NTFS Junctions to share directories between test suites!)
- GUID
Bob reaps the advantages of having all his usable tests on top when he goes to make a test list! He creates a dynamic list and adds tests by his directory (1 Admin Tests\Item Creation\). He uses numbers to avoid the hassle of alphabetical sorting so that it's easy to find the actual test suites on top of his list. Finally, he has his library on the bottom as these tests almost never need to be edited.
// Fin
Let me know if this gives you any insight on how you might solve your own problem. It was fun to write!
-Robert
//////// Q('.'Q) Greg's Message Q('.'Q) /////////
Hi Robert,
We are data driving as much as we can. That's actually the source of the dilemma.
Here's a typical scenario. Set up a "driving" test, we've found this works best for us. That test calls a login test, Step A test, step b test, test C test step D test then log off test.
Except for login test none of these tests can be run without some other test in front of then to leave you at a particular screen. Leaving all these tests in one directory will lead to a terrible mix of tests that can be run alone and ones that can't. A totally unworkable mess once we get more than a few tests going. So, the driving tests need to be in one place and the others in some other place. Ideally we would know which driving test calls which driven test, either based on organization or a feature that will tell us which tests are called by the given test.
Thanks for your suggestion.
Gregg.
/////////// Q('.'Q) End Greg's Message Q('.'Q) /////////////
Gregg,
I always have my steps organized in separate directories from my tests. Their sub directories mirror the actual tests so that it's easy to find which test goes with what, or they reflect in what piece of the puzzle they fit.
Example Scenario:
Bob wants to write a test to create an item for his hipster shopping site on the admin site, then check to see if that item appears. There are many different ways to configure an item, and certain combinations create different results. He has 4 pages to navigate to in order to setup the item.
Page A: Properties - Defines the name of the item, it's unique item code, and the type of item. After this, you click "next"
Page B: Inventory - Defines the number of items in stock, and the minimum and maximum a person can order at once. After this, click "next"
Page C: Pricing - Define the price of your item, the shipping cost or, optionally, check a box that says the item is FREE with another purchase. After this, click "next"
Page D: Category - Define which category the item will be displayed in. After this, click "create and preview"
There are many ways Bob wants to test these pages. First, an item with the type "unappealing shirt" can not be marked "FREE". Second, the maximum number of "classy hats" is always 10. And finally, items that are FREE are always automatically added to the "SALE" category in addition to whatever the user defines.
Bob realizes that some parts of his test are reusable. Since his environment isn't fresh each time, he navigates to an online GUID generator using his library of universally useful testing scripts to generate a unique ID for his item. Next, he uses a step required in most tests for his site - a login test to his admin portal in order to add items. These tests are named "GUID" and "LOGIN".
Bob then creates a data driven test to add in the name and unique ID from his extracted GUID from his earlier test, then select an item type from the drop down menu using a data driven element which he reads from his spreadsheet in column "ItemType". This test is named "Page A".
Next, he creates a two tests for Page B. One test enters all the values (maximum, minimum, inventory) from his spreadsheet. He calls this test "Page B Default". His second test only adds data for the minimum and inventory from the spreadsheet. This test checks to make sure that the maximum was automatically set to "10" as defined by the requirements above. He calls this test "Page B Classy Hat".
He creates three tests for Page C. His "Page C Default" test data drives the price, ship cost and ignore the "Free" column. His second test "Page C Free" does what the default test does, but also checks the "Free" box which brings up a dialogue box requiring him to select what items the person needs to purchase to get that item free. Finally, he has "Page C Unappealing Shirt" test, which ensures that the "FREE" checkbox is disabled.
His two tests for page D either select a category from his data (Page D Default) or ensure that "SALE" is selected, as is required for anything "FREE" (Page D Sale).
Phew! That's a lot of tests! Mixing and matching these will provide Bob flexibility in running his tests. He may have one go through all the defaults, one with "FREE" selected, but otherwise all defaults, etc. Problem is, none of these tests run properly by themselves as they're all pieces of a series of tests which will make sure that each permutation of the item creation works.
Knowing that Test Studio Test Lists are formed from a screen that organizes them top to bottom, he opts to place the steps in directories as follows. Tests are in BOLD. Directories are underlined
1 Admin Tests
- Item Creation < This is where the "macro" level tests are stored
-- Default Item Test
-- Classy Hat Test 1
-- Classy Hat Test 2
-- Unappealing Shirt Test 2
2 Admin Test Steps
- Item Creation Tests < This is where the pieces are stored. Note: the directory has the same name as the tests that call these
-- Page A
--- Page A Default
-- Page B
--- Page B Default
--- Page B Classy Hat
-- Page C
--- Page C Default
--- Page C Sale
--- Page C Unappealing Shirt
-- Page D
--- Page D Default
--- Page D Sale
- Common Tests < This is where tests that could run in any test
-- Login
3 Library < Common tests for all his projects are stored here. (Tip: Use NTFS Junctions to share directories between test suites!)
- GUID
Bob reaps the advantages of having all his usable tests on top when he goes to make a test list! He creates a dynamic list and adds tests by his directory (1 Admin Tests\Item Creation\). He uses numbers to avoid the hassle of alphabetical sorting so that it's easy to find the actual test suites on top of his list. Finally, he has his library on the bottom as these tests almost never need to be edited.
// Fin
Let me know if this gives you any insight on how you might solve your own problem. It was fun to write!
-Robert
0
Gregg
Top achievements
Rank 1
Rank 1
answered on 09 Sep 2013, 02:45 PM
Hi Robert,
Thank you for such a detailed and well thought out answer. It is exactly what I am looking for. I had put together part of the solution but not enough of it so it would scale as our tests, and projects, expand.
Now to go over it with my supervisor and the other Telerik developers so we can all be on the same page. <G>
Gregg
Thank you for such a detailed and well thought out answer. It is exactly what I am looking for. I had put together part of the solution but not enough of it so it would scale as our tests, and projects, expand.
Now to go over it with my supervisor and the other Telerik developers so we can all be on the same page. <G>
Gregg
0
Hi,
Ivaylo
Telerik
@Robert, thank you once again for your input and helping others. I have updated your Telerik Points.
@Gregg, I am happy to hear that you found the right solution. If you need any additional assistance, please do not hesitate to contact us.
Ivaylo
Telerik