I am testing a site that has 21 major pages (modules) some of which contain embedded pages (modules).
I’m in the process of refactoring my large tests. I’ve broken out steps that are common to each test into their own test file (e.g., environment setup, sign-in, etc.).
I’ve also broken up each primary module into its own separate test file which, along with the common tests, I’m treating as separate functions.
I then use each of these tests to create unique tests using “Test as Step” in the Step Builder.
Thus far, I have been successful in getting this to work for some of my tests.
The problem I’m having is that when a test (function) does fail (e.g., Page 8), I can’t set a breakpoint or “Run to Here” command on that file because only the selected page is executed (i.e., none of the common setup tests that are required to get to the page 8 test are executed).
I’m stuck and wonder if missing a key step, or whether there’s a better approach to handling this type of work.
Any suggestions or help would be greatly appreciated.
Thanks,
Dan
8 Answers, 1 is accepted
Your approach is quite intelligent and I guess it will be much easier for you to maintain your tests. Please note that 'Run To Here' executes all steps including the test as steps. This will ensure your test reaches the failure step and you will be able to debug.
Another quite useful feature that you might find suitable for your scenario is that the recorder could be attached to an existing browser instance. Though you should keep in mind this is available for IE only.
Please let me know if you would need further assistance! Thank you!
Regards,
Elena Tsvetkova
Telerik by Progress
Rank 1
Hi Elena,
Thanks for the response. I was hoping to find a way to execute a Run-to-Here operation against a sub-file (function), but after having given that further thought, I'm not sure how that would be expected to work if the function doesn't know what was the calling function (file).
I didn't know about the ability to attached to an existing browser instance, and while it may take more time to navigate to the deeper portions of the product, I think this solution may work.
Thanks again,
Dan
Rank 1
Hi Elena,
After working with this a little more, I'm finding that I'm still limited as to what I can do using separate files as part of the "Test as Step" feature/functionality.
As I noted in my last response, the "Run to Here" functionality works only on the Main script (i.e., it doesn't work if you've set that on a separate file that's being used in a "Test as Step" capacity). This is understandable, and am guessing due to the lack of persistence of "Run to Here" commands.
I am surprised that Breakpoints don't work on "Test as Step" files, given their persistence (i.e., they exist until the breakpoint is removed). In my case, I have a "main" script that calls a "Test as Script" file which contains a breakpoint. The main test successfully calls the Test as Script file, but when the line containing the breakpoint is executed, the breakpoint is ignored and the script continues to run to completion or until an error is encountered. Is this a bug, or is there a way for Test Studio to recognize the breakpoint in "Test as Step" files?
Also, I've found that files saved and run as "Test as Step" don't exihibit the same behaviors found when running a single, self-contained test file, or a "Main" file that contains "Test as Step" commands the references separate files. In the attached screenshot, I show how the editor window looks when showing the execution of the main file. It has a yellow header and icons showing the status of the run (Pass|Fail|Not Run). When I click down into the "Test as Step" lines, the subsequent edit windows don't render the headers or icons, making the debugging process more difficult than need be. The user can open the log file and work through that, but it would make for a more consistent user experience and assist with script debugging.
The screenshot illustrates the current behavior and how it might look if these elements were to cascade through all files or "Test as Step" commands.
1. The Pass|Fail|Not Run icons are not rendered as part of the test pass in the editing windows.
2. The yellow header showing the count of lines and lines executed is not rendered for these files.
You should consider that Test Studio does not offer that advanced debugging tools and features. The features I recommended are available but do not offer that many features compared to Visual Studio for example. Though you could export Test Studio project into VS and use its functionalities for better debugging experience.
Here are my comments on your remarks:
- breakpoints in a child test are not considered by execution of main test - this is known issue for us but is still under construction and investigation; there is a feedback submitted into our public portal
- both the Pass|Fail|Not Run icons and the yellow header are only available for the main test; if a sub-test fails it is only indicated that this step has failed but the exact details are only available in the execution log. Double clicking the red cross opens the test file itself ready for edit.
Another feature that Test Studio could offer for debugging is the Visual Debugger. I hope you will find the best approach for you to be able to consistently maintain your scripts. Thank you for your understanding!
Regards,
Elena Tsvetkova
Telerik by Progress
Rank 1
Hi Elena,
Thanks for the feedback and suggestions.
Dan
I hope you will find the best approach for your scenario. Should you have further queries do not hesitate to contact Test Studio Support Team!
Regards,
Elena Tsvetkova
Telerik by Progress
Rank 1
I have bumped into this awkwardness as well.
For example, I have a test which contains only two test steps. Let us say that the first has 10 steps and the second has 20 steps. If there is an error when the test runs, the only way I see to work is to:
1) create a new temporary test.
2) put the steps from both test steps into it, so that I have a new 30-step test.
3) run and fix the test.
4) figure out what I had to change and apply those changes back into the two test steps.
This last step is particularly quirky and prone to missteps and errors. I end up entering a cycle where I think I have put in all the changes but the original test still does not work. So I have to do (4) again, see what I have missed and keep iterating on that until it works.
Not fun.
And this is with a test that only has 2 test steps. If I end up with more complex tests, how bad is this going to be?
I have seen that I can go to a terminal and do:
grep '"Description"; `find * -name \*tstest`
I could write a script that would read this is and allow me to compare the steps in two tests so that I can see what is changed or added and so on. But really? Like a perl or awk script? What the heck? Which millennium is this?
But I am using the Test Studio IDE. I has no re-factoring tools
I am pleased to see you found some time to explore our forums as well and I will be happy to provide some additional information on the topic.
At first I would like to confirm that the request to implement breakpoints also for nested tests is completed and available in the product since the beginning of this year. So, you will be able to use the Visual Debugger against that parent test which only contains test as steps - set a breakpoint in the nested test, ensure you have set the debugger options to stop in quick execution mode and run the test.
In addition to that you can use the partial execution options from the step context menu Run... -> To Here, From Here, Selected steps. So following the example you described, you can go for the following sequence of actions:
- open that parent test with two tests as steps
- here, you can either choose the first, or the second, depending on the current needs and then Run... -> To Here
- this will trigger the test execution in the listed order and will run the steps until any of these fails, or the last step is executed. Then the recorded will be attached to the browser in its current state.
- here you can continue recording new steps - these will get recorded in the active test in the Test Studio project.
- or you can open any other test, which contains steps against that current state of the page and execute any of its steps using Run... -> From Here, or Selected Steps.
- to get back to the example - let's the quick execution fails on the last step of the first sub-test, you can open that sub test in the Test Studio project and work directly on it and its elements against the current active recording session. Once you fixed that error, you can open the second sub-test to be active in the Test Studio project and continue the execution using Run... -> From Here again using the same active recording session.
I hope this will be useful for you as well. However, if you need any further assistance, please do not hesitate to get back to me.
Regards,
Elena
Progress Telerik