I have a Parent test that executes a Child test in a coded step:
[CodedStep(@
"Step1"
, RequiresSilverlight=
true
)]
public
void
Step1()
{
ExecuteTest(
"Child.aii"
);
}
I did some debugging and checked the test result in OnAfterTestCompleted() method in both Parent and Child. In Child the status was "Fail" but in Parent it was "Pass".
Is there any setting that could cause such behavior or is it simply a bug?
PS I just checked and the problem doesn't occur if I execute the Child step as a regular, non coded step (by using Add > Code as step). If I transform this into a coded step again it stops working. This suggests a bug...
7 Answers, 1 is accepted
Hello Filip,
Please try using the following change in your code for running a test as a step:
[CodedStep(@
"Step1"
, RequiresSilverlight=
true
)]
public
void
Step1()
{
this
.ExecuteTest(
"Child.aii"
);
}
Please let us know if you experience the same issue using the above instead. If this does not resolve the issue, are you able to add a test using the normal "Test As Step" option from the Add menu, then right-click the step and choose Customize Step in Code? We'll look for your reply with the above.
Greetings,
Keaegan
the Telerik team
Hi Telerik
Adding "this." to my code didn't help.
Also, in the last part of my first post I already wrote that I tried adding test as step manually (Add -> Test As Step). Such step works ok (i.e. the entire test fails if the step throws an exception). However, after I transformed it into a coded step (using Customize Step in Code) it stopped working again although I did no modifications to the generated code.
Cheers
Filip
I have tried to reproduce this in both our 2010.3.1213 build and our 2010.3.1307 build. Both versions bubble up the exception thrown by the subtest to the parent test which correctly marks the overall result as Failed.
Overall Result: Fail
------------------------------------------------------------
'1/27/2011 4:24:12 PM' - Starting execution....
'1/27/2011 4:24:14 PM' - Detected custom code in test. Locating test assembly: PruebasTestBrickControl.dll.
'1/27/2011 4:24:14 PM' - Assembly Found: C:\Users\gibson\Documents\Support Issues\PruebasTestBrickControl\PruebasTestBrickControl\bin\Debug\PruebasTestBrickControl.dll
'1/27/2011 4:24:14 PM' - Loading code class: 'PruebasTestBrickControl.WebAiiTest2'.
------------------------------------------------------------
'1/27/2011 4:24:15 PM' - 'Pass' : 1. Navigate to : 'http://www.telerik.com/'
'1/27/2011 4:24:17 PM' - 'Fail' : 2. [WebAiiTest2_CodedStep] : @"Execute test 'WebAiiTest1'
>>> Test-as-Step 'WebAiiTest1.aii' log starts:
Overall Result: Fail
'1/27/2011 4:24:17 PM' - 'Fail' : 1. Verify 'TextContent' 'Contains' 'Not Automated Testing' on 'AutomatedLink'
------------------------------------------------------------
Failure Information:
~~~~~~~~~~~~~~~
Content.TextContent of 'AutomatedLink' does not match!
Match Type: 'Contains'
Expected Result: 'Not Automated Testing'
Value at time of failure: 'Automated Testing'
------------------------------------------------------------
'1/27/2011 4:24:17 PM' - Detected a failure. Step is marked 'ContinueOnFailure=False' aborting test execution.
------------------------------------------------------------
'1/27/2011 4:24:17 PM' - Overall Result: Fail
'1/27/2011 4:24:17 PM' - Duration: [0 min: 0 sec: 148 msec]
------------------------------------------------------------
<<< Test-as-Step 'WebAiiTest1.aii' log ends.
------------------------------------------------------------
'1/27/2011 4:24:17 PM' - Detected a failure. Step is marked 'ContinueOnFailure=False' aborting test execution.
------------------------------------------------------------
'1/27/2011 4:24:17 PM' - Overall Result: Fail
'1/27/2011 4:24:17 PM' - Duration: [0 min: 2 sec: 802 msec]
------------------------------------------------------------
'1/27/2011 4:24:17 PM' - Test completed!
What excepting is being thrown by the subtest? Can you send me a test that repros the symptom that I can study?
Kind regards,
Cody
Because of my company policy I can't send you the failing test. Since this only happens in this particular child test (other children tests work fine) I can't even recrete that with a sample test :\
I'm just wondering if you can think of any tests elements or settings that could be causing such behaviour?
Cheers
Filip
Can you share what exception is being thrown by the subtest? Do you have a stack trace you can share? I can try to replicate that exact same exception and see if it repros the symptom.
No there are no settings or elements that would be able to cause this behavior. I even tried setting "ContineuOnFailure" on the step that calls the subtest. I still could not reproduce this symptom.
Cody
Thank you for the repro steps. You're right I don't need your tests/files when I am able to reproduce the problem. I asked Filip for his test files so that I can figure out how to repro this problem.
The problem you are running into is that "this.ExecuteTest" doesn't return any results or do anything in anyway to abort the parent test. To overcome this please don't call child tests via code. Use our Test-as-step feature and call your child tests via our standard test step. It's cleaner, requires no code, and the results from the child test will cause the parent test to abort.
Another option is to use our RunHelper class something like this:
ResultSummary results = ArtOfTest.WebAii.Design.Execution.RunHelper.Test(@
"c:\TestProjectDir\NoResults.aii"
, @
"c:\TestProjectDir\bin\debug\TestProject.dll"
);
if
(results.Passed ==
false
)
{
throw
new
ApplicationException(
"Child test failed"
);
}
In the databound parent test scenario you will still be faced with the parent test running 500+ times. We do this deliberately so that the tests will return results for all rows of the table. Most of our customers do not want the test to abort on the first row that failed. They want to know if the rest of the rows pass/fail as well. Greetings,
Cody
the Telerik team
Have you looked at the new Online User Guide for Telerik Test Studio?