NavigateURL does not work inside an ELSE statement

[Shashi]

Telerik team,

It looks like a Navigate statement is not working correctly when inside an IF/ELSE statement.
 
To reproduce, create a new test, record the following statements and then run the test.  The IF clause can be anything as long as the test gets into the ELSE clause. 

IF (Wait For <entity> Exists)
ELSE
   Navigate to site.

Behavior is as follows:
1) Test connects to the site successfully but the navigate step hangs after that.
2) In Quick Execute mode, the Quick Debugger window (window that shows the current step executing and buttons to pause, step, etc) disappears after a while but the test does not terminate.  I have sometimes seen the word "Failed" very briefly just before this happens.
3)  In Test view, the test does not continue (or terminate if the above is the only code in the test) after connecting to the site.

Other information:
1) This problem is specific to the latest release of Test Studio (2012.1.1411.0).  This construct worked without issues in the previous version of Test Studio that we were on (2011.1.825.0).
2) Problem is reproducible against any site - I was able to reproduce it against the Google home page as well as our own Silverlight web application.
3) Navigate works fine if it is not inside an if/else statement.
4) Problem occurs both with a recorded Navigate statement as well as a coded step (call to ActiveBrowser.Navigate).
5) I have not tried this but you may also be able to reproduce the problem if you place the Navigate step inside the IF.

This is a serious issue for us for the following reason:
We have the above construct in almost all of our 100+ shared tests.  We need to do this in order to run our shared tests in stand-alone mode - which we need to do in order to debug them (breakpoints in shared tests are not honored when they are called from parent tests).  Changing the way we do this is a massive undertaking that we simply do not have the time for at this point.  So I am hoping that you will be able to reproduce the problem - and provide a solution/fix pretty quickly.

Let me know if you need further information.  I hope to hear from you soon.

Thanks,
Shashi

[Byron]

Hello Shashi,

I was not able to reproduce your issue in your version of Test Studio. We will need additional information before we can isolate your problem. If possible, please provide access to your application and a copy of your test project. If you cannot provide these, please provide a test project that exhibits the behavior against a public site. Also, please provide exported test execution and failure logs for your test and an application log for the behavior. Thanks for helping us better identify your issue.

Byron
the Telerik team

[Shashi]

Hi Byron,

A sample test (with associated resx and cs file) is attached.  Please add them to a VS 2010 project, build and run.

I look forward to hearing back from you.

Thanks,
Shashi

[Byron]

Hi Shashi,

Thanks for providing the additional information I requested. I was able to reproduce your issue using the test file you sent me using our newest build, although I have not been able to reproduce this in any test not incorporating the test you sent me. I found that removing the coded step or changing the if condition resolved the issue, which may help you in resolving your specific issue. In the meantime, I have filed a bug report on this issue, the status of which you can monitor at our Public Issue Tracking System (PITS).

Regards,
Byron
the Telerik team

[Shashi]

Hi Byron,

I tested before I added the coded step (with just the recorded step) - and it was hanging in that scenario as well.  When the coded step was added, I tried executing the recorded step first - that hung as well.

What changes did you make to the 'if' statement to get it to work? 

Thanks,
Shashi

[Byron]

Hi Shashi,

The test does hang after only removing the coded step. However, inserting another step (like a navigate) before the if...else step resolves this. For the if condition, I changed the condition to a verify against a public site (which I knew would fail in order to trigger the else branch). I recorded a video demonstrating these possible workaround directions.

Regards,
Byron
the Telerik team

[Shashi]

Hi Byron,

I can't access the video because the site you posted it on is blocked by my company's IT department.  Can you provide a listing (of the two workarounds) - or attach the test file (with all associated files) - in this thread itself? 

Thanks,
Shashi

[Byron]

Hello Shashi,

1. The first method was to change the verification of the if step. I used a verification against Google that I knew would fail in order to trigger the else branch.

2. The second method was to add a navigate step before the if step and remove the coded step in the else branch.

I have attached screenshots of both solutions.

Kind regards,
Byron
the Telerik team

[Shashi]

Byron,

Thanks for your response and suggestions.

A couple of comments/questions:

a) Adding a Navigate at the top of the test is not an option for us. 
The tests where we have this construct are shared tests - which are called from other tests.  Idea is that the shared test expects the application to be in a certain state when called from the parent.  The 'if' statement tests for the existence of an element that is supposed to exist when the application is in the expected state - if the check fails, the shared test does whatever is necessary to get to that expected state (this is the code in the ELSE statement).

Given the above, you can imagine what would happen if a Navigate statement were to be added at the top of the shared test - especially when shared test is called from a main test.  Test would switch to a different page in the middle of the test - and then we would need to add code to get it back to the expected state (which may not be easy in all cases - not to mention that it shouldn't be necessary to do this).     

b) Regarding changing the condition of the 'if', does your test work if you do the following:

i) Delete the Navigate before the 'if'.
ii) Check for existence instead of visibility

I tried changing the condition of the If (Wait For Exists) in my test to an element that is on the Google home page - then it worked (without navigate ahead of it).  I am seeing this in some of my older shared tests which have a different check in the IF (element is not necessarily unique to the state that the shared test needs to be in) and those work in stand-alone mode. 

That puzzles me - why should it make a difference what I check for in the 'if' statement? 

Note - the only reason we are doing all this is to be able to debug shared test code at runtime - which we cannot do when it is called from main tests.  Are there any near-term plans to support these features in the UI?  By debugging, I am referring to functionality such as breaking at statements in the shared test, stepping into and out of shared tests, stepping through shared test code, etc. 

Thanks,
Shashi

[Byron]

Hi Shashi,

The navigation step was only necessary if I kept the if condition you provided. If I changed the if condition, no navigation step was necessary. My test also works if the condition is an 'exists' verification rather than a visibility verification. We are still investigating the cause of the behavior your test sample exhibits.

Although the breakpoint test step property is not respected within a test as step, you can use the Execution > Pause Test Step Property instead. Support for stepping into and out of shared tests and stepping through shared test code is planned, but we do not have a definite timetable for implementation.

Kind regards,
Byron
the Telerik team

[Shashi]

Hi Byron,

Thanks for the information and the tip on the alternate way to debug steps inside shared tests (I will check the latter out).

I am currently looking at the feasibility of changing the test conditions of the shared tests that fail.  While doing this, I came across a different but related issue.  In some of our shared tests, the code to handle stand-alone operation (navigate to the site, log in and do whatever is necessary to get to the expected state) is inside an IF (Wait for Exists Not) instead of IF(Wait For Exists)).  The code assumes that the "Wait For Not Exists" will succeed in stand-alone operation and fail when the test is called from another test. In other words, I am assuming that the Wait For Not Exists behaves like a Verify Not Exists when it is inside the IF clause. 

This was working in the previous version (2011.1.825) - but it does not work in Test Studio 2012 R1.  In the latter, Wait For Exists Not seems to be failing in stand-alone operation.  So when I run the test stand-alone, it skips the code to navigate, log in, etc.  Was this a planned change in 2012 R1 or any of the 2011 versions after 2011.1.825?  

I know how to fix the issue - change all IF(Wait For Exists Not) to IF (Wait For Exists) with the code to handle stand-alone operation going to the ELSE clause.  But there are a number of shared tests that I will need to update and I want to be sure this is required before I make the changes.

Thanks,
Shashi 

[Shashi]

Byron,

I just discovered that the editor now allows me to change Wait For Exists and Wait for ExistsNot to a Verify Exists and Verify ExistsNot, respectively (we did not have this ability in the previous version we were on).  If I do this and then convert the Verify statement into an IF statement, the code seems to work correctly. 

I am continuing to have the issues with Wait For Exists and Wait For ExistsNot described below - and I look forward to your findings on those issues.  However, if we do not encounter any further issues, converting these to Verify Exists and Verify ExistsNot would be an acceptable solution (since that is what we really want to do).

Thanks,
Shashi

[Shashi]

Byron,

I noticed you created Support ID 547850 for the last reported issue - thanks for doing that.  I have sent you the sample you requested - I look forward to your feedback.

We are going to move forward with the plan of using IF (Verify Exists) and IF (Verify ExistsNot) to get us around the shared test problems we are having. However, one of my team members has discovered a couple of minor issues with that approach as well.  Neither of these issues are show-stoppers for our plan - but I am reporting them nonetheless so your team can take a look at them. 

Here are the steps to reproduce for the two issues:

Issue 1
1.  Run the shared test stand-alone.
2.  Break at the IF (Verify Exists) or IF (Verify ExistsNot).
3.  Manually navigate to the site and page so that the element (that shared test is detecting in the IF statement) exists.
4.  Continue running the test.

Expected:  The test should behave as if the element (that the IF statement is looking for) exists.  If the condition is IF (Verify Exists), it should execute the code inside the IF (if any) and then move on to the rest of the code outside the IF.  If the condition is IF (Verify ExistsNot), it should execute the code inside the ELSE (if any) and then move on to the rest of the code inside the IF.

Actual:  The test behaves as if the element does not exist - behavior is exactly the opposite of what is expected.

Issue 2:
1. Run a test that calls the shared test.
2. Break at the step that calls the shared test.
3. Manually log out of the application (or do something so that the element that the IF statement is detecting) does NOT exist any more.
4. Continue the test.

Expected:  The test should behave as if the element (that the IF statement is looking for) does NOT exist.  If the condition is IF (Verify Exists), it should execute the code inside the ELSE (if any) and then move on to the rest of the code outside the IF.  If the condition is IF (Verify ExistsNot), it should execute the code inside the IF (if any) and then move on to the rest of the code inside the IF.

Actual:  The test behaves as if the element exists - behavior is exactly the opposite of what is expected.

II am hoping you can repro at your end - try a Silverlight app if you can't repro with a non-Silverlight app.  Let me know if you cannot repro.

Thanks,
Shashi

[QA]

 

Hi,

My colleague, Shashi, had reported this issue a while back. I am having the same issue.

I used If/Else Verify ExistNot and have Navigation URL inside the IF statement. However, there is a delay before the Navigation URL is typed in to load the site causing huge performance delay when running 100 + tests. Then I reduced the timeout to 1sec for the If Verify condition but I don’t think Test Studio looks at that (Is this a known issue?).

Then, I replaced Verify with Wait for Exist in IF/ELSE loop. The page loads faster but the test hangs on Navigation step.

Please let me know if the issue that Shashi reported has been fixed and if so, in which Test Studio version.

Thank You,

Dipti

[Byron]

Hello Dipti,

The original issue which was reported by Shashi is marked as resolved and its fix was included in Test Studio build 2012.1.719 (see version notes, PITS#11175).

If reasonable for your situation, I suggest upgrading all the way up to our latest internal build. However, if you do require an older build of Test Studio standalone, please see the 'Older Versions' section from your account (or this link for Test Studio Express).

Could you please confirm the version of Test Studio you are using, as well as what specific timeout value you are referring to which is not being respected on execution? Also let us know how you are running the test (eg. via Quick Execute or Test List, etc).

Regards,
Byron
the Telerik team

[QA]

Hi Byron,

Thank you for your respose. We are in the process of upgrading to the latest internal build if everything goes well.

To answer your question:

Could you please confirm the version of Test Studio you are using - [2012.1.411.0]

as well as what specific timeout value you are referring to which is not being respected on execution? - [it is called 'Timeout']

Also let us know how you are running the test (eg. via Quick Execute or Test List, etc).- [Quick Execute, Test List and Test View]

Thank You,
Dipti

[Shashi]

Byron,

While evaluating the latest official version of Test Studio (2012.2.1420.0) for possible upgrade, I have encountered an issue with regard to IF(Wait for Exists).  It seems that it does not behave correctly if the element it is waiting on already exists.

We have a few tests that were handling different versions of our software when they were first written.  These have a structure like this:

IF (Wait for Exists (element in new version) ) then
   <execute steps appropriate to new version>
ELSE
   <execute steps appropriate to old version>
ENDIF

This code is executed correctly by our current version of Test Studio (2012.1.411.0) - but in the new version, the code goes into the ELSE and then hangs.  NOTE:  Both old and new versions are running against the same version of the application - which is the new version.  Also, in all cases where I encountered this issue, the element already existed on the page.

Is this new behavior by design - if not, is this a known issue?  If the latter, is a fix available or is there a plan to fix it any time soon?

I realize that I can easily work around the issue in this case by removing the IF/ELSE clause and only having teh code for the new version (which is what I am currently doing).  But it may not be as easy to fix in other scenarios.

Thanks,
Shashi
p.s.  I am reporting this issue in continuation of this thread since it could be related to the original issue of this thread.  Please let me know if you would prefer me to start a new thread for this.

[Shashi]

UPDATE:  I am also seeing a similar problem with WHILE (Wait For Exists) - if the element already exists, the steps inside the while clause were not executed.  I tried changing the statement to WHILE (Verify Exists) - this resolved the above problem but then the steps inside the WHILE were executed even if the element did not exist.

Any input would be much appreciated.

Thanks,
Shashi

[Byron]

Hello Shashi,

I apologize for the delay. I was not able to reproduce this behavior. It is possible that the behavior you observed is due to a change in the find logic behavior for Test Studio: between 2012.1.411 and 2012.2.1420, Test Studio's find behavior was updated so that steps which cannot locate an element using the primary find logic will fail, even if they are located by backup find logic. In earlier versions, if an element could be located using the XPath find logic, the step would pass with a warning in the test execution log.

If my hypothesis is correct, then Wait steps targeting these elements should fail whether or not they are the logical conditions for an IF or WHILE block; if Wait verifications succeed against your target elements on their own, but only fail as conditions, we will need more information. Please provide us with a copy of your test and access to your application. Also, please provide your failure log and a copy of the DOM at the time of failure. You can export these to file from the failure tab for your test and add them to a .zip archive. If the test is passing, you can export an execution log and a DOM at the time of the problem step using our Visual Debugger. If it is not possible to grant us access to your application, please take a Fiddler trace and attach it to a support ticket in a .zip file. You can capture a Fiddler trace using FiddlerCap. A Jing video of the behavior may also help us diagnose your issue. Thanks for helping us diagnose your issue.

Regards,
Byron
Telerik