Facing issues while Running a set of Test Cases in a Test List Test Studio scheduling

10 posts, 0 answers
  1. rashmi
    rashmi avatar
    11 posts
    Member since:
    Feb 2016

    Posted 08 Mar Link to this post

    Hi team,

    We are evaluating Telerik test studio as a Test Automation tool for Automation testing for one of the products for our customers . The customer product is a Windows Application build in C# and WPF and uses DevExpress (3rd Party) for UI Controls.
     Currently we are carrying out  Proof of Concept (POC) to automate few business scenarios in the product.  We are able to automate some of the scenarios and it runs successfully in quick execution without any error.

    But during test list scheduling,some test cases fails with "Null reference exception". 


    I have a list with 40 test cases. But I see that isn't stable, b/c sometimes it run correctly and sometimes the test fail(approx.38/40). Not sure why is the inconsistency on it.And today if one fails,next run some other test case will fails.

    So, not sure why they are failing in the list but individually it passes consistently.

    Please any one help me on this.

  2. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 11 Mar Link to this post

    Hello Rashmi,

    Thank you for your interest in Test Studio.

    Please note that when executing WPF tests remotely your application should use the same path as on your local machine. Can you confirm this? I am not clear are you able to execute any tests on the remote machine at all. If some times are getting executed sometimes not, I will need you to provide me with the execution log so I can take a look.
    If non of the tests are executed remotely and you are using the same path for your WPF application as on the local machine, then I will need you to provide me with the application log. Here is a short video on how to get the application log from the remote machine.

    Looking forward to hearing from you.

    Regards,
    Ivaylo
    Telerik
     
    The New Release of Telerik Test Studio Is Here! Download, install,
    and send us your feedback!
  3. rashmi
    rashmi avatar
    11 posts
    Member since:
    Feb 2016

    Posted 11 Mar in reply to Ivaylo Link to this post

    Hi Ivaylo,

    Thanks for replying.

    I am executing/scheduling test cases in local system only and I am using correct WPF application path.

    I am able to schedule all test cases successfully without any error. But the thing consistently I am not getting 100% results(mostly I am getting 95% pass rate).

    Some of the test cases fails while clicking buttons in the wpf dialogs.But in quick execution it pass successfully,only in test list execution it fails that to not consistent.

    I have tried enabling 'UseStepWaitOnElement' parameter on each step. But still some test cases are getting failed.

    It looks like 'UseStepWaitOnElement' parameter and Wait.ForExists(20000) (code behind method) is not working?

    Thanks,

     

  4. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 15 Mar Link to this post

    Hеllo Rashmi,

    Please provide me with the execution log so I can review the error and advise further. Please note that the execution in a test list is quicker than in standalone test execution as you may already noticed. You can try replacing the wait for exist with a fixed wait and see whether it will work for you.

    Looking to hearing from you.

    Regards,
    Ivaylo
    Telerik
     
    The New Release of Telerik Test Studio Is Here! Download, install,
    and send us your feedback!
  5. rashmi
    rashmi avatar
    11 posts
    Member since:
    Feb 2016

    Posted 21 Mar in reply to Ivaylo Link to this post

    Hi Ivaylo,

    Please find the attached log for the failed test case when we run on test list. I have tried putting waitforexist as below,

    .WelcomeTOMTextblock.Wait.ForExists(2000);
    .WelcomeTOMTextblock.SetFocus();
    Assert.IsFalse((ArtOfTest.Common.CompareUtils.StringCompare(WelcomeTOMTextblock.Text, "Welcome "+expectedText, ArtOfTest.Common.StringCompareType.Same) == false), string.Format("Verify \'WelcomeTOMTextblock\' text Same \'Welcome test\' failed. Actual value \'{0}\'", WelcomeTOMTextblock.Text));

    This reduced the failing percentage but still not 100%.And also most test cases are failing while clicking 'Ok' button in wpf dialog window.

  6. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 23 Mar Link to this post

    Hello Rashmi,

    Could you try using a fixed wait instead of wait for exist and give it a try. Also if the failing step has the WaitForNoMotion property checked (refer to the screenshot attached), try to uncheck it and run the test again.

    Regards,
    Ivaylo
    Telerik
     
    The New Release of Telerik Test Studio Is Here! Download, install,
    and send us your feedback!
  7. rashmi
    rashmi avatar
    11 posts
    Member since:
    Feb 2016

    Posted 31 Mar in reply to Ivaylo Link to this post

    Hi Ivaylo,

    Thanks for reply.

    Giving fixed wait reduced the test case suite failures at some extent.But giving these fixed wait for every step increases the execution time .

    There is an global test list setting called ElementWaitTimeout ,Does this work with trial version?

    Because I can see execution will be hanged(only during suite run) if it expecting the element(like 'ok' button) inside wpf dialog and if the dialog loaded slowly even though its before the timeout period what we have set,its not able to find the element.

    And also I can see for every step there is an setting called 'UseStepWaitOnElement',if we enable the setting 'Wait on elementsTimeout' will be set to 30 sec.

    Does these above settings work with trial version because even though I am enabling the setting,its not serving the purpose.

    Can you please tell me how to set the Element waitForTimeout.properties.

    Thanks,

  8. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 04 Apr Link to this post

    Hello Rashmi,

    "Giving fixed wait reduced the test case suite failures at some extent.But giving these fixed wait for every step increases the execution time ."

    That is the idea, to set the fixed wait to specific steps, where needed, without adding a wait step on every other step. Setting global timeouts will definitely increase the test execution, which you don't have to do.

    If you continue experiencing problems you can provide us with access so we can investigate.

    Regards,
    Ivaylo
    Telerik
     
    The New Release of Telerik Test Studio Is Here! Download, install,
    and send us your feedback!
  9. rashmi
    rashmi avatar
    11 posts
    Member since:
    Feb 2016

    Posted 11 Apr in reply to Ivaylo Link to this post

    Hi Ivaylo,

    I have listed the issues facing below while running test list,

    1.Most test cases failing while finding elements when application is navigated to new page. --> Is there any method/properties to wait for the wpf window to load like wait(window name) etc.

    2.While validating title textblock text also its failing 

    3.I am getting null reference exception in the following code,

    Applications.Uiexe.ApplicationWindow.Find.RefreshRoot(); 

    is there any other method which refresh the wpf dom tree.

    4.During suite run,execution will be hanged if its not able to find the some element .Any idea why its not continue with next test case.

    Can you please help me to solve the above issues and make my test list run 100% consistent.

    Looking forward to your reply.

    Thank you,

  10. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 14 Apr Link to this post

    Hello Rashmi,

    All the issues you are getting are because your tests need improvements. By only listing the problems you are experiencing I am not able to help much. You are the person testing the application you should know its specifics best and should consider where to use wait, where to refresh the DOM etc. 

    Regarding refreshing the DOM try one of the following methods:

    Manager.ActiveApplication.MainWindow.VisualTree.Refresh();
                 
    Manager.ActiveApplication.MainWindow.RefreshVisualTrees();

    Regarding the element not found you should investigate whether the element is not found because a timing issue (need bigger wait) or this is because the find logic is not correct (i.e. a dynamic attribute is used).

    Regards,
    Ivaylo
    Telerik
     
    The New Release of Telerik Test Studio Is Here! Download, install,
    and send us your feedback!
Back to Top