Assert Fail and continue question

[CVue]

I have one huge coded step test. I have some codes and assert fail/continue statements inside this coded step. My continue on fail is:

try
{
     Assert.IsTrue(....)
}
catch (Exception e)
{
      Log.WriteLine(e.Message);
}

..... keeps running until all codes are run.

 

Seems like after the step is run, the the coded step passes, but I know there is a assert statement inside the coded step that fails and was printed to the log.writeline. Is there anyway to fail the coded step if one of the assert statement fails but still continue to run the whole coded step?

[Cody]

Hi Cher,

The only way for a coded step to be marked Fail is to allow the exception to happen and let Test Studio run time engine deal with it. By catching it the way you are you are hiding the exception from the Test Studio run time engine.

What you could do instead is set a flag in your CATCH section, then throw a new exception at the bottom of the code if the flag is set:

catch (Exception e)
{
      Log.WriteLine(e.Message);
      failStep = true;
}
.
.
.
if (failStep)
      throw new Exception();

Regards,
Cody
Telerik

 

 

[CVue]

Ok that works, but what if I want to throw that exception and print out what was the fail(s) that follows the new exception thrown at the end of the codes instead of having to find through the log which assert failed?

[CVue]

Or should I say, how can I throw a exception for each assert fails at the end of the coded step, Let's say if i got two assert fails, when i go to if(failStep)..... the log will only show that fail part, but if i had another failed, it will not show that exception message. I want to show both on the log.

[Cody]

Hi Cher,

"how can I throw a exception for each assert fails at the end of the coded step" To be honest you can't. This is a technical impossibility. Now having said that do you really need to? Inside every CATCH block you are logging the failure which is added to the cumulative test log. What's wrong with throwing just one generic exception at the end of the coded step then viewing the contents of the logs for the additional information? If you have 2+ exceptions that are caught and logged in your CATCH blocks all of them will be visible in the test log after the test run.

Regards,
Cody
Telerik

 

 

[CVue]

how many exception can you throw? After doing this to more than 30 tests, i'm getting this error

 

ArtOfTest.WebAii.Exceptions.ExecuteCommandException: ExecuteCommand failed!
InError set by the client. Client Error:
System.Runtime.InteropServices.COMException (0x80028019): Old format or invalid type library. (Exception from HRESULT: 0x80028019 (TYPE_E_UNSUPFORMAT))
   at mshtml.HTMLDocumentClass.IHTMLDocument3_get_documentElement()
   at ArtOfTest.InternetExplorer.IECommandProcessor.SetCurrentDocumentMarkup(BrowserCommand& response)
   at ArtOfTest.InternetExplorer.IECommandProcessor.ProcessInformationCommands(BrowserCommand request)
   at ArtOfTest.InternetExplorer.IECommandProcessor.ProcessCommandInternal(WebBrowserClass ieInstance, BrowserCommand request, IHTMLDocument2 document)
BrowserCommand (Type:'Information',Info:'DocumentMarkup',Action:'NotSet',Target:'null',Data:'',ClientId:'Client_c168ab9f-204a-400e-81af-417440858fac',HasFrames:'False',FramesInfo:'',TargetFrameIndex:'-1',InError:'True',Response:'System.Runtime.InteropServices.COMException (0x80028019): Old format or invalid type library. (Exception from HRESULT: 0x80028019 (TYPE_E_UNSUPFORMAT))
   at mshtml.HTMLDocumentClass.IHTMLDocument3_get_documentElement()
   at ArtOfTest.InternetExplorer.IECommandProcessor.SetCurrentDocumentMarkup(BrowserCommand& response)
   at ArtOfTest.InternetExplorer.IECommandProcessor.ProcessInformationCommands(BrowserCommand request)
   at ArtOfTest.InternetExplorer.IECommandProcessor.ProcessCommandInternal(WebBrowserClass ieInstance, BrowserCommand request, IHTMLDocument2 document)')
InnerException: none.​

[Cody]

Hi Cher,

how many exception can you throw?

Basically one per test. The first exception thrown in a coded step will cause the test to immediately stop and abort that test run. Of course you can have multiple coded steps and each one can have a "throw" statement. The first throw will cause the rest to be skipped and ignored.

If talking about a test list, there's no technical limit. Every single test in your test list can have a "throw" statement. It's not possible for this to cause a "ExecuteCommandException" (assuming you did not use RecycleBrowser).

For us to get to the bottom of what's causing your "ExecuteCommandException" we need a way of reproducing the problem in our source code debugging environment. Can you send us a complete test project we can run that will reproduce this problem?


Regards,
Cody
Telerik

 

 

[CVue]

I don't have that issue anymore. but back to your snippet,

 

catch (Exception e)
{
      Log.WriteLine(e.Message);
      failStep = true;
}
.
.
.
if (failStep)
      throw new Exception();

 

I have a exception for each tests that if any fail, it will throw new exception when all the code have run through. I've run through all my tests and none supposedly failed, but why is it still throwing a exception ?

[Cody]

Hi Cher,

I don't know the answer to that question. I'd have to see it happening first hand on my computer so that I could diagnose it and determine the root cause.

One simple thing that could cause such a behavior is if at the time the variable "ailStep" is declared it was not initialized to false. If it got initialized to true, then the "if (failStep)" would always execute what's in the if block.

If that's not it I'll need a method to reproduce the problem so that I can study it.


Regards,
Cody
Telerik

 

 

[Shashi]

Cher,

In case you are not aware, all Assert statements (at least those in the ArtOfTest.Common.UnitTesting namespace) have an override with a parameter for an error message - this message will be output to log if the Assert fails.  For example, Assert.IsTrue has 2 overrides:

public static void IsTrue(bool condition);
public static void IsTrue(bool condition, string message);

If you haven't already done so, you may want to check those out and see if you can use these to accomplish what you ​want to do.   

 

But that said, ​I have a couple of comments that you may want to consider:

a) I am not sure I see the point of using Asserts if you are going to catch every exception - ​seems to me you are defeating the purpose of using Asserts and making your code unduly complicated in the process.  If you ​are catching the exception so that the test does not fail, then ​a better way to implement it would be to not use Assert statements at all - instead, use simple if/else statements (use Log.WriteLine to log information and data to the log).  Asserts are designed to be used when you want the test to fail if the condition is not met - yes, there are certain conditions under which you would wan the assert to fire and then catch the exception (but that should be rare in tests - more common in application code where you don't want the user to see every exception that is thrown).

b) Regarding the desire to continue on fail, the root cause of all your issues may be your design of having "one huge coded step test".  I am assuming that means your test has only one step - which is a large coded step with many asserts in it.  If so, I am not sure why you decided to implement it that way - but that is not an optimal test design because it is not easy to figure out where a test failed and why - without a lot of troubleshooting and digging around in the log file.  You may be better off if you can break up that coded step into multiple small coded steps (each testing one condition).  When you do this, don't catch the exception in these coded steps - instead, turn on Test Studio's Continue on Fail flag on those steps where you don't want the test to stop if they fail.  This would make your code a lot simpler and smaller because you then wouldn't have to handle exceptions.  You can use the first suggestion above to log assert failures to the log file.

 

Hope that helps.  Please forgive me if I have misunderstood what you are trying to do.

Shashi

p.s. I am not a Telerik employee but a long time user of Test Studio (specifically of the VS addin).