(I am just starting to play around with remote test execution and there isn't a lot of clear documentation, so I may simply be missing the basics.)
I have a VM configured as a Remote Execution Server and it is successfully running basic tests when initiated from within Test Studio and the Scheduling Server. In the last couple of weeks I have been redoing some tests to make them simpler and more flexible when handling errors, and in doing so I have copied some Coded Steps from other existing tests. This has ended up in a scenario where many of my tests will not run on the Remote Execution Server because they cannot find the elements I have added to the Element Library. To clarify, the elements exist in my library on other tests, but it seems that because they are not local to this specific test, the elements cannot be found. NOTE: The whole project runs great locally on Test Studio; it is only the remote run that fails to find the elements.
If I record on that test and click on the respective elements, creating steps that I don't need, then the test seems to run remotely without error. However, I will have to do this for hundreds of elements and it will be quite time-consuming. Is there a way to mass transfer project files / elements to the remote execution server so it knows what they are?
Similarly, I also noticed that when I duplicate a test to modify it and create a different scenario, it only brings over the elements that are created in Test Studio steps. Anything referenced in a coded step (i.e., Actions.SetText(Pages.WebsiteName.ElementName, text)) does not bring over the element, so if I later want to create a Test Studio step using this element, I have to re-record it in order to recognize it in the local test. Trying to select the item from "Elements" and use the "Step Builder" doesn't help, as it only shows the "Getting Started with the Step Builder" text when I go to Actions or Verifications. If I record that element once in the local test, then Actions and Verifications will display their options. Is there a way to bring elements into new tests without having to re-record them or manually copy Test Studio steps that include the elements?
(I hope I'm using the right terminology...)
Thanks ahead of time!
Regards,
Steven Klassen
5 Answers, 1 is accepted
Thank you for the information provided on this case.
The problem with not present element during the remote run as described is an expected one. I suppose that the elements from coded steps refer to missing elements using Pages notation. The recommended approach for stand alone executed tests in not to use elements recorded in other tests. To escape such problem a find expression could be used to locate element and then apply an action on it in a coded step.
For group of tests that would be executed in same test list it could be applicable since all elements definition would be present in frame of the test list to be send for remote execution.
Unfortunately there is no general approach to resolve at once this problem. A work around could be to have this project in TFS source controlled environment and to use option "Get latest version of tests automatically from TFS" when schedule a test list. This would assure that on the remote machine latest project version would be uploaded before the execution and no compilation exceptions would occur.
It is possible to predefine elements to have them in Elements repository if you need to have them there - not recording them in the standard way.
I hope this information is helpful.
Best Regards,
Nikolay Petrov
Progress Telerik
Rank 1
Hi Nikolay,
Thank you for your response. I appreciate the details, and I think I have a better understanding of how Test Studio functions, although I'm not sure how the predefined elements would work. It seems like they require the full manual work of a coded Find step, but my coded Find steps would be more easily searchable and editable.
While this is an answer to my question, it does make me concerned about the long-term viability of Test Studio as a full test suite. I am finding that most of the things I want to do in a test now require coded steps, but the one thing that was particularly faster (and easier) through Test Studio's UI was the Find Element logic Test Studio implements. Unfortunately it now seems like I have to use coded look-ups even for this to work around Test Studio's limitations with remote execution. The primary purpose is to be able to run the tests remotely on our test environment, but I can't use UI-based Find Element steps at all unless I do them manually in each test that needs to use a particular element. This drastically increases the time to create additional tests. It seems that a browser-neutral (and machine-neutral) Selenium-based code solution might be more effective in final production.
We do not have a TFS implementation, although we do have a variety of Git-based solutions. Is there a workaround using Git with Test Studio to enhance remote execution?
Extra question: Does Test Studio summarize this kind of information (recording elements vs. doing them in code when remotely executing tests) into best practices anywhere, or is this explained in the paid training courses? I have encountered a variety of speed bumps like this along the way and I'd like to minimize those as much as possible, if my understanding of the product is the limitation.
As a best practice in sense of locating and interacting with elements in code I could point to use find expression to locate an element but not using predefined Pages notation. This way the coded step could be used in different test with no remote compilation exceptions.
Would you share more details on the case where the UI generated find expression could not be in service? I think that a sample would be better for us to understand the problem and for you - to have a working approach. If it contains private data you could open a support thread instead the forum post.
In general the most of the automation steps should be recorded in the product UI and in very few - related to a specific custom scenario - to use code to complete the job.
Unfortunately the remote execution with Git source control option - to automatically use latest files is not available yet.
I believe you could find quite useful our training courses in terms of better understanding the work-flow automation process in Test Studio. There is a lot of information available for the product features and guidance for common problems in our documentation portal as well.
Best Regards,
Nikolay Petrov
Progress Telerik
Rank 1
Hi Nikolay,
I am considering the training courses, but I would need to know that the features exist that way I am expecting before I pay money to learn how to use them. My main question is the difference (and limitations) when using Test Studio's Find Element vs doing everything with code-based Find Expressions. I'm fine doing everything in code, but then there is less purpose for me to use Test Studio rather than a free Selenium-based API for testing. Test Studio provides some great built-in features that I use regularly with Pages notation: OnInput events, Clicks, SetText, and more. However, I can only use these features if I have a step in that test which defines the element.
Is there no way to include the element reference in a test without having a placeholder step that interacts with that element? Do I have to include dummy steps to keep using it in code or in remote execution?
For example, in one of my tests I have to keep a couple of element steps like "Verify 'MyElement' Exists" and then uncheck the box so the step doesn't run, just because I want to use 'Pages.Site.MyElement' in code to generate the values, run OnInputs, use SetText, and click at certain programmatically-determined times. This seems tedious, and while I can do all of this with straight code, it is reducing the usefulness of Test Studio's built-in API and tool features.
Basically, I feel like I have to do more and more things with code rather than with Test Studio, and if that's the case I don't want to pay more for training. I'd like to use the tool to the most efficiency, but I don't see it filling a lot of the gaps right now.
Let me try to clarify the main point here. General purpose of Test Studio is to save time in generating automation tests using the product UI. There should be very few cases where additional actions from the user could be necessary. Since the UI can not cover each one custom application implementation - the Testing Framework background of Test Studio could serve to cover great amount of possible scenarios.
In this context recording tests should be mainly done from the Test Studio UI. Once there is a scenario that could be completed in code it is recommended to keep the code as simple as possible to have as less relations between test as possible. This is why Pages notation is not recommended way to work with elements. It should do the job if the tests are executed always locally since the entire project is compiled before execution.
The simple answer to your question is - no, there is no way to refer element defined in one test to be used in other (Pages.Site.MyElement) and to execute the second test remotely with no compilation exceptions.
I believe that there should be more "in Test Studio" way to complete the scenario need. The main purpose of trainings would be to reveal most of the product features and to propose an approach to common automation problems using the tool.
Best Regards,
Nikolay Petrov
Progress Telerik