No results on failed test

[Joseph]

Hi,

Im having a problem with test studio (Trial version). i have experienced this problem before but the most recent case is the most annoying.

Over the weekend i attemted to run a test using a data source of 1000 new users, looping through testing each stage of the login procedure, the test runs fine i have run in 1 - 20 successfully, so i thought i would run a bulk import and try that, however somewhere along the way the test failed (110 - 115) and test studio seems to have stopped running the test completely.  However test studio didnt produce any results, i ran this test in a test list, and when i look at the calender there are no tests on friday, even though i started the test friday at 5:15,

Apart from this being incredibly annoying its also making it extremely hard to error check my tests.

Is there anyway to force test studio to produce results on failed tests? or does it have to run 100% of the test and the datasource to calculate any results?

[Plamen]

Hello Joseph,

I am sorry to hear you're running into this problem. A test with that many data iterations is very likely to encounter memory issues deep into execution causing the beahvior you're describing.  To overcome this, please try using our ReuseAppWindow feature available in the Test Properties. This defines the number of iterations to reuse the browser window for a single test. 
 
By default, zero keeps the same application open during the entire data driven test causing high memory consumption. Increasing that number can help to improve memory and performance. If you change it to 50, for example, the browser will close and reopen after each fiftieth iteration, which will reset the memory consumption of the browser.

If you continue to have difficulty, please send us the Application Log. Ensure you enable logging first (clear out existing log, then run the problematic test) and send us the log within a .zip file.

Regards,
Plamen
Telerik

[Joseph]

Thanks for replying, I will try that tonight and see how it reacts.

One question though, if I have a data source of 100 new users to be added, and the test closes and reopens the browser on every 50 when the test continues to the second 50 will the data source be reset and start from 'user 1' again? or will it go straight to 'user 51' for example?

Thanks

[Plamen]

Hi Joseph,

It will not reset your data source. It will only reset the browser in order to release the memory taken by its process and then it will continue with iteration 51. Please see this article for more information.

Regards,
Plamen
Telerik

[Joseph]

Hi, Sorry I haven't replied, its been a busy week. I have tried this solution and it started off well and got stuck after about 28 records ( I set the ReuseAppWindow to 10)

Is there a way to close the browser completely? rather than just opening another? I can see a problem arising when I start entering 100+ users and having the system open many browsers, if for example its set to ReuseAppWindow to 10 or even 20

Thanks

Jo

[Plamen]

Hello Jo,

I am sorry to hear you're still having problems. Please try to workaround the issue by adding a static iteration counter variable and a coded step at the end of your test(as a very last step), which will close the browser after a certain iteration and reopen it.

// Add your test methods here...
static int iterationCounter=0;

[CodedStep(@"New Coded Step")]
publicvoidWebTest1_CodedStep()
{
    iterationCounter++;

    if(iterationCounter == 20)
    {
        this.ActiveBrowser.Close();
        this.Manager.LaunchNewBrowser();
        this.Manager.ActiveBrowser.WaitUntilReady();
        iterationCounter = 0;
    }
}

Please try the code above and let me know if the issue persists.

Regards,
Plamen
Telerik

[Joseph]

Hi Plamen,

I have tried the piece of code you suggested it seems to work ok, but I need to clarify something as its doesn't seem to be working as I expect.

for example I have 100 users (databound) that my test runs through inserting each one, I have set the new code to 10, so my understanding was that it would run in 10 users close the browser - reopen the browser and insert the next 10 and so on. is this accurate?

the reason I ask is because on test 35/100 I received an error code from my project ( a popup informing the user to contact support, a standard message that forces the user to press ok) this is no fault of test studio but our system, now of course this was an unexpected stage in the test so it failed and as the pop up was still visible the next step failed also,
 
however I expected the tests to continue to fail up to the 40th test then close the browser/reopen browser then continue with the reminder of the tests. this is not the case though, I have looked through the log and it just seems to have failed every test after 35 without attempting to close the browser

Any help on this would be great

*Update: I just realised that the problem is probably that if the test is failing its possible the close/reopen browser step is never getting reached? is it possible to use ReuseAppWindow and the new code together? how will the test react?


Jo

[Plamen]

Hi Jo,

You should be able to use ReuseAppWindow and the coded step together. The coded step will be executed on every iteration as a regular test step which will close and reopen the browser. Then when you reach the iteration set to the ReuseAppWindow  property, the browser will simply close and reopen one more time at the end of the iteration. 

A better approach is to add a conditional dialog handling coded step in your test where you expect the Alert dialog to appear. See this article for more information. In this way if the dialog appear Test Studio will handle it and continue with the next test step without causing a failure.

Regards,
Plamen
Telerik