I have made several tests with no issue. However, on playback , it seems to be a different story. For instance, I will play a test which fails at Step 25. I mark it as Run to Here and start it again, so I can investigate. This time, the test fails at Step 16 and never reaches 25. This has happened to me several times, but have not been able to figure out the issue.
I am currently working on a wizard navigation piece that spans across 2 domains. At the halfway point, the first domain (abc.domain.com) redirects to another server with the second domain (xyz.domain.com). In case it matters. The tests seem to fall apart after the domain switch.
Rank 1
Iron
1 Answer, 1 is accepted
Hello Steve,
I am sorry to hear you are having troubles with the test run consistency and will be happy to assist you with this.
Based on the description you shared, it seems that the test runs need to be synchronized with the application responsiveness when executing the steps. The mechanism for this in Test Studio are the wait and verification steps, as well as execution delays. These steps help you control the speed of test step execution at the time of the run so that the actions are sent, when the application under test is ready to accept these. The idea is that the page under test may be loading slower on particular machines, or at particular moments - and you need to allow the necessary time for the page to fully load, before executing the next action.
So, implementing wait steps in between the recorded steps will help to reduce such random failures. The common advise where to include the waits/verifications is when you expect any change on the page - reloading the page completely, or retrieving some information from the server, which is dynamically displayed (a good example here could be the autocomplete dropdown, which values gets filtered as you type).
Once you modify the tests, you should see improvement in the remote test runs and consistency in the returned results. Still, you may encounter sporadic failures due to the page responsiveness speed at certain moment. If you rerun the test, though, this will be executed just fine. For such occasions, we have implemented a test list setting, which allow you to rerun the failed tests automatically.
I hope that these suggestions will help you improve the test execution results. Let me know if you need any further assistance on the topic.
Thank you for your cooperation in advance.
Regards,
Elena
Progress Telerik