Hi All,
I am trying to find out a way to skip a telerik test.I found some available asserts in Artoftest.common.Unittesting . But i couldn't find any assert for ignore/skip a test step.
Can you let me know how can we skip a test using assert.
I know to skip a test using condition, i am looking for a way so that like pass/fail, skipped tests numbers also will shown in the custom report/ logfile as skipped / Not run.
Strangely, in the execution extension interface provided by telerik team, i found a Notrunsteps parameter.
Currently skipped tests are shown as passed.
Thanks,
VVP
13 Answers, 1 is accepted
Asserts can only be used to cause a test to fail and abort the test at that point due to the assert condition not being met. The Notrunsteps represents steps that were skipped because they were disabled, or skipped because the other part of an IF/ELSE block was executed.
Please clarify do you want to skip specific steps of a test or an entire test? I ask because these two statements appear to contradict each other:
"I am trying to find out a way to skip a telerik test." "But i couldn't find any assert for ignore/skip a test step."
Before venturing further trying to explain how to do either one, what is it you're trying to accomplish with this? What automation challenge are you faced with? There may be a better approach to solving that problem.
Regards,
Cody
Telerik
Rank 2
Hi Cody,
Sorry if my post was confusing.
I meant to say skip the test step only. Not the entire test.
Scenario is like this. There are let's say 10 steps. First one is login. If login itself fails, i want to skip the remaining steps.
So currently i am setting a flag in step 1 and if flag is true then only i am going inside remaining steps.
From execution point of view , things are happening correctly. When step 1 fails, no other step will get executed.
But in execution report,all are showing as passed. I am looking for a solution for that issue.
Thanks,
VVP
When a test step fails every step following it is supposed to be marked "Not Run" as shown in the two attached screen shots. You have stated "in execution report,all are showing as passed". This is not right or normal. We need to get to the bottom of why this is happening in your tests. Please share with me the test log or results showing everything as passed as you describe. This will be more fruitful then trying to figure out how to force steps to be skipped since our runner is supposed to be doing that for you already.
Regards,
Cody
Telerik
Rank 2
Hi Cody,
1)
I tried scenario where i didn't set "ContinueOnFailure=True" and made to fail first step.I was able to see like you have shown in screenshot. But in logs no information about other steps (whether passed /failed /skipped) are printed.
Logs - AutomationLogs.txt
2)
Another scenario where i have around 10 steps
-> First is login and if it fails( i fail steps through Assert.Fail in Visualstudio unittesting), i am just skipping steps using "If" loop
like
if (exeflag == true)
{
// Perform Step operation
}
else
{
Assert.Inconclusive("Skipping test); // using Microsoft.VisualStudio.TestTools.UnitTesting;
}​
See the logs attached for this. AutomationLog_Steps_ShownAsPassed.txt
Kindly let me know if you need more info.
Thanks,
VVP
But in logs no information about other steps (whether passed /failed /skipped) are printed.
That is the expected behavior from Test Studio when ContinueOnFailure=False. Since Test Studio aborted the test at the failed step, it's not possible to give a Pass/Fail status since there's no information to give.
I guess I'm confused on what you want to see now. It sounds like you want the test to stop on the failed step but at the same time run the rest of the steps to get pass/fail information. Those two are mutually exclusive. Please clarify how you want the test to be executed.
Please do not use Assert's from Microsoft.VisualStudio.TestTools.UnitTesting in Test Studio coded steps. They are not compatible with the Test Studio framework.
I'm beginning to think the problems you are having are being caused by incorrect use of ContinueOnFailure, and maybe a misunderstanding what you're supposed to get when this is set to false. Please explain how you want your test results presented so we can figure out the right way to get there.
Regards,
Cody
Telerik
Rank 2
Thanks for the info.
I replaced Visualstudio unit testing asserts with Telerik Asserts. Only reason why i used Visual Studio ones is because it offers Assert.Inconclusive(); which is similar to Nunit Assert.Ignore().This is what i am looking for in Telerik Test studio.
As i went through your mail, i believe there is no mechanism for skipping a test,We have to set ContinueOnFailure=false and Telerik will abort the steps.
In my scenario what i was trying to achieve is described below.
Lets assume, i keep ContinueOnFailure=True meaning even if first step fails, it will proceed to next steps.
I am executing through ArtofTest.Runner and not through UI.
-> 10 steps are being executed,
-> 1st step( If Login fails, ArtOfTest.Common.UnitTesting.Assert.IsTrue(LoginSuccess) will fail the step and set a bool exeflag=false.)
-> Remaining 9 steps has a If condition in the body, which checks exeflag
--> if exeflag=true, then only execute the remaining steps.
--> if exeflag=false, I need to skip and have some information in logs mentioning thes ewere skipped.
So Can you suggest any solution for my scenario.
Thanks,
VVP
Cody will update this ticket shortly.
Thank you for your patience.
Regards,
Ivaylo
Telerik
Only reason why i used Visual Studio ones is because it offers Assert.Inconclusive(); which is similar to Nunit Assert.Ignore().This is what i am looking for in Telerik Test studio.
I understand the desire. Unfortunately Test Studio does not understand this assert and will not display the information you intended it for.
As i went through your mail, i believe there is no mechanism for skipping a test,We have to set ContinueOnFailure=false and Telerik will abort the steps.
Basically correct. We do have the conditional IF/ELSE block but the condition must be some property of the web page being displayed e.g. if text ABC is present then run the IF block else run the ELSE block. The condition cannot be whether or not some previous test step passed or failed.
So Can you suggest any solution for my scenario.
To be very honest and direct, don't set ContinueOnFailure=True. Setting it to true is the cause of the issue you're trying to fix. By leaving this setting =False Test Studio will detect the login failed, abort the test and not even try to run steps 2-10.
Now if your Login step is passing when it should not be (allowing execution to incorrectly go to step 2) that's a different problem we can solve. The Login step needs to have some sort of verification to verify that login was successful e.g. the "Log out" button/link appears. This will make the Login step (which is commonly subtest) correctly fail when it is supposed to.
Regards,
Cody
Telerik
Rank 2
Hi Cody,
Sure. I will not set ContinueOnFailure=True for first step. So if login fails, remaining steps will be skipped.That will work as partial solution.
As per my understanding, we dont have explicit mechanism for skipping tests right.
1) So can we raise a feature request or something to include in future releases.? It will definitely help .
2) When tests are skipped (ContinueOnFailure=True is not set and login fails) , that information should be made available in logs. Currently only through UI, we will be able to make out right.
Thanks,
VVP
When you look at the results displayed by our Results view (see attached screen shot) we do show # of steps that passed and total # steps in the test. Are you asking for additional information beyond this? Can you give me a more specific example what you would prefer to see?
Regards,
Cody
Telerik
Rank 2
Hi Cody,
Yes i have used Telerik execution extensions and printing pass,fail information to a text file.
But i am unable to get Skipped steps. If i get number of skipped steps, then report will be complete. In UI also it is not shown right.
Thanks,
VVP
Thanks. I created this feature request for you. Did I word it correctly?
Regards,
Cody
Telerik
Rank 2