DataDriven based testing best practices

20 posts, 0 answers
  1. Mercedes
    Mercedes avatar
    6 posts
    Member since:
    Jul 2012

    Posted 01 Nov 2012 Link to this post

    Hi,
        I have a  test studio script that is data driven by a SQL server database. The web application against which the script was recorded is a policy system. The script works fine for new business system but i would like to know the best process to follow in creating a script that can be used for policy changes.

    For example, if i have a set of entries on the new policy, the system would create a new policy but when there is only 1 entry change in the above set of entries, i do not want the complete script to execute but just the field that has been changed. The reason for that is, i have several hundred fields of data entry in the application and only one value could be changed.

    Thank you
  2. Cody
    Admin
    Cody avatar
    3354 posts

    Posted 06 Nov 2012 Link to this post

    Hi Brian,

    I apologize for the delay getting back to you on this.

    What sort of policy are we talking about here? An insurance policy? A business policy? An HR policy? An advertisement policy?

    Let me reiterate the problem just to make sure I understand it correctly:
    1) When you create a policy there are several hundred fields that must be filled in, then the policy is saved to the system
    2) You want to test making changes to an already existing policy stored in the system
    3) You want to test changing just a few (or one) fields and do not want to have to re-enter all of the fields

    Do I understand correctly?

    Part of the problem here is how do you identify which field needs to be changed i.e. how can the test detect that field 2 or field 202 needs to be modified? Is there something in your data source that will specify this? Another detail to work out that is part of the same question is whether or not there are multiple pages involved e.g. fields 1-50 are on page 1, fields 51-99 are on page 2, so if somehow the data tells the test to set field 79 to X we need to know we have to navigate to page 2 in order to get to field 79.

    Let's start by defining what the input data will look like when changing a policy. Will this input always include all data for all input fields, or will it somehow try to tell the test "change field n to X"? The test implementation varies greatly based on this answer. We'll go to the next step once you answer this starting question.

    Kind regards,
    Cody
    the Telerik team
    Are you enjoying Test Studio? We’d appreciate your vote in the ATI automation awards.
    Vote now
  3. Mercedes
    Mercedes avatar
    6 posts
    Member since:
    Jul 2012

    Posted 07 Nov 2012 Link to this post

    The policy i am talking about here is an insurance policy. What you specified is right. I have the input data for the new policy setup but for the changes i have not decided on how to do it as there are several fields on several pages. I needed to know what the process is to compare two records in a database so that i can run the script based on that.
    If there is no way to compare two records on the database, i think it would be better to have all the fields in the record included and modify just the field that needs to be changed. That way the script will run through each step and pick the same values again and modify just the field that has changed. This would take longer time for the script to run which is the reason i wanted to find out better options. Please let me know if you have any questions.
  4. Cody
    Admin
    Cody avatar
    3354 posts

    Posted 11 Nov 2012 Link to this post

    Hello Brian,

    I needed to know what the process is to compare two records in a database...

    Test Studio does not have any built-in functionality for direct DB access or manipulation. We do have this code sample demonstrating how you can write your own code to access a DB. It would be possible (and not all that hard) to modify this code sample to compare two records in a DB and return the differences. You just have to come up with a SQL query to do so. Or you could perform two SQL queries to get two records in memory and use code to compare the two records.

    Now the part I don't understand is what two DB's or records are you going to compare to get the differences? It's easy to see you have the main DB where the new policy has been created and saved. But what exactly will you compare against?

    Greetings,
    Cody
    the Telerik team
    Are you enjoying Test Studio? We’d appreciate your vote in the ATI automation awards.
    Vote now
  5. Mercedes
    Mercedes avatar
    6 posts
    Member since:
    Jul 2012

    Posted 14 Nov 2012 Link to this post

    Hello Cody,
          Thanks for taking the time to provide a solution. We have decided to run through the complete script and let test studio pick the values even if the record hasn't changed. This could cause the script to run longer than manually performing the actions but it would help us perform the work unattended.
           An unrelated question regarding my script. I might have to provide you a demo to properly explain this but let me try. I have an OK button on a web application, which when clicked gives me a pop-up. So i have a step in the script that clicks the OK button and then performs a dialog handle but when i am running the script, i noticed that the script executes the "click the OK button" but for some reason keeps searching for the button even after it has been executed. Can you please suggest why this is happening if you have seen similar issue before or i could provide you any information you need?

    Thanks
  6. Cody
    Admin
    Cody avatar
    3354 posts

    Posted 15 Nov 2012 Link to this post

    Hi Brian,

    Can you send me a screen shot of this dialog, or even better a video of this happening? Jing is a good free screen recorder. My one suspicion is that this dialog is a modal dialog. Modal dialogs have a bad habit of not returning control back to the test script until the modal dialog is closed. For this reason we normally set SimulateRealUser to true on the click step that causes the modal to open. This prevents the test from hanging indefinitely inside the click step.

    All the best,
    Cody
    the Telerik team
    Are you enjoying Test Studio? We’d appreciate your vote in the ATI automation awards.
    Vote now
  7. Mercedes
    Mercedes avatar
    6 posts
    Member since:
    Jul 2012

    Posted 15 Nov 2012 Link to this post

    Hi Cody,
        I recorded a video while playing back the script. I inserted a breakpoint, so that it will be easier to analyze the steps in a step-by-step fashion. There is a submit button which when clicked should display the pop-up that states that "it is an out of sequence transaction", and then the confirm dialog should click on ok. In the recording, you can see that the submit button by itself is clicking on the submit button and the pop-up OK button also and then the next step which is the actual confirm dialog is waiting for the dialog to appear. Please let me know if you have any questions.
             I did not see the option to set SimulateRealUser under the properties for either of the steps. The only option remotely close was to set SImulateRealClick to true which i tried unsuccessfully.


    Thank you
  8. Cody
    Admin
    Cody avatar
    3354 posts

    Posted 15 Nov 2012 Link to this post

    Hi Brian,

    My apologies, it's SimulateRealClick as shown in the attached screen shot (not SimulateRealUser... however that property does exist for the Set Text test steps).

    I watched your video (had to find and download the correct CAMTASIO codec to play it) and am a bit concerned by what I see. Here's what I'm observing in the middle:

    1) The test clicks the OK button for the Client Access Endorsement screen
    2) This causes the JavaScript Confirm dialog to show up
    3) Then this dialog goes away. I am concerned what caused this dialog to close. I find it hard to believe that Test Studio closed it simply because the test step still running was the one to click the OK button for the Client Access Endorsement screen.
    4) Then the app proceeds to the next screen but now Test Studio is trying to handle the Confirm dialog that's already come and gone. It's out of sequence.

    I'd like to look at this problem on your computer via GoToMeeting. Let me know what time zone you are in and your availability and I'll setup the meeting. I'm in Austin, TX which is Central Time (GMT -6).

    All the best,
    Cody
    the Telerik team
    Are you enjoying Test Studio? We’d appreciate your vote in the ATI automation awards.
    Vote now
  9. Mercedes
    Mercedes avatar
    6 posts
    Member since:
    Jul 2012

    Posted 16 Nov 2012 Link to this post

    I am in Eastern Time, Indiana. I am available from 1PM - 4 PM. Please let me know the time, so that i can share my test with you.
  10. Cody
    Admin
    Cody avatar
    3354 posts

    Posted 16 Nov 2012 Link to this post

    Hi,

    All right. Let's target 2pm Central (3pm Eastern):

    1.  Please join my meeting, Friday, November 16, 2012 at 2:00 PM Central Standard Time.
    https://www2.gotomeeting.com/join/688837290

    2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

    Dial +1 (805) 309-0012
    Access Code: 688-837-290
    Audio PIN: Shown after joining the meeting

    Meeting ID: 688-837-290

    GoToMeeting®
    Online Meetings Made Easy™

    Not at your computer? Click the link to join this meeting from your iPhone®, iPad® or Android® device via the GoToMeeting app.


    All the best,
    Cody
    the Telerik team
    Are you enjoying Test Studio? We’d appreciate your vote in the ATI automation awards.
    Vote now
  11. Cody
    Admin
    Cody avatar
    3354 posts

    Posted 16 Nov 2012 Link to this post

    Hi Nithin,

    Just to document the outcome of our meeting:

    1) We discovered you're trying to do conditional dialog handling based on the contents of the data source. You were conditionally calling a subtest. This approach doesn't work. The problem is that the dialog handler must be initialized and watching for the dialog before the dialog appears. Using the approach you had the dialog would appear, then the sub test would get control and initialize the dialog handler. By then it's too late since the dialog is already open. The subtest will ignore already open dialogs.

    2) You had multiple Handle Confirm dialog handlers in your test. There is a known limitation in Test Studio... you can have only 1 dialog handler of a specific type in a test. You can get around this by using sub-tests to handle the additional dialogs of the same type as needed. We fixed your test today by splitting out the affected steps into sub-tests

    3) We also created a coded step that will find a policy row based on the policy number, then find the machine Modify button and click on it.

    Regards,
    Cody
    the Telerik team
    Are you enjoying Test Studio? We’d appreciate your vote in the ATI automation awards.
    Vote now
  12. Mercedes
    Mercedes avatar
    6 posts
    Member since:
    Jul 2012

    Posted 29 Nov 2012 Link to this post

    Hi Cody,
           I tried using the steps that you created for me to find a policy row based on the policy number, then find the modify button and click on it. I had to make some changes and am getting the error "Index was out of range. Must be non-negative and less than the size of the collection."  I tried looking to see why the error was generating but could not figure it out. Any help is appreciated.

    Thanks
  13. Cody
    Admin
    Cody avatar
    3354 posts

    Posted 29 Nov 2012 Link to this post

    Hi Brian,

    It's impossible to say what is causing this without seeing the code involved and being able to run it and study the behavior on my machine. How about we look at this together in another GoToMeeting? On Friday I have only my 10:30 slot open. On Monday I can do 1 hour anywhere from 10:30 - Noon (Central time).

    All the best,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  14. Mercedes
    Mercedes avatar
    6 posts
    Member since:
    Jul 2012

    Posted 30 Nov 2012 Link to this post

    Cody,
          Friday at 10:30 am Central Time should be fine with me.

    Thank you,
    Nithin
  15. Cody
    Admin
    Cody avatar
    3354 posts

    Posted 30 Nov 2012 Link to this post

    Hi Nithin,

    You got it:

    1.  Please join my meeting, Friday, November 30, 2012 at 10:30 AM Central Standard Time.
    https://www2.gotomeeting.com/join/452673762

    2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

    Dial +1 (510) 443-0605
    Access Code: 452-673-762
    Audio PIN: Shown after joining the meeting

    Meeting ID: 452-673-762

    GoToMeeting®
    Online Meetings Made Easy™

    Not at your computer? Click the link to join this meeting from your iPhone®, iPad® or Android® device via the GoToMeeting app.


    Greetings,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  16. Cody
    Admin
    Cody avatar
    3354 posts

    Posted 30 Nov 2012 Link to this post

    Hi Nithin,

    To summarize our meeting today:
    1) We determined the wrong code was being used in the test having the problem. You will fix this yourself.
    2) We looked at how to handle a case where a search returns "no client found" message. The conclusion was that you need to click on the first "select" button whether or not a client is found.
    3) I showed you how to verify that text is set to Read-Only.

    I am now closing this ticket as the main issue is resolved.

    All the best,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  17. Mercedes
    Mercedes avatar
    6 posts
    Member since:
    Jul 2012

    Posted 05 Dec 2012 Link to this post

    Log.WriteLine("Form Action is: " + Data("FormAction"))
    If Data("FormAction") = "Add" Then
        Me.ExecuteTest("User Access\FormAdd.tstest")
        'Remove the dialog from the monitor
        Manager.DialogMonitor.RemoveDialog(ad)
    Else If Data("FormAction") = "Modify"
         Dim formcell As HtmlTableCell = ActiveBrowser.Frames(0).Find.ByExpression(Of HtmlTableCell)("InnerText=" + Data("FormNumber"))
         Log.WriteLine(formcell.ToString())
         'Dim formcell As HtmlTableCell = ActiveBrowser.Frames(0).Find.ByExpression(Of HtmlTableCell)("InnerText=" + "PL 0002 0909")
         Dim row As HtmlTableRow = formcell.BaseElement.Parent.[As](Of HtmlTableRow)()
         Dim modifyBtn As HtmlInputButton = row.Cells(6).Find.ByExpression(Of HtmlInputButton)("value=Modify")
         modifyBtn.Click()
         'Remove the dialog from the monitor
         Manager.DialogMonitor.RemoveDialog(ad)
         Me.ExecuteTest("User Access\DeleteFormSchedule.tstest")
         Me.ExecuteTest("User Access\FormDetail.tstest")
    Else If Data("FormAction") = "Remove" Then
         Dim formcell As HtmlTableCell = ActiveBrowser.Frames(0).Find.ByExpression(Of HtmlTableCell)("InnerText=" + Data("FormNumber"))
         Log.WriteLine(formcell.ToString())
         Dim row As HtmlTableRow = formcell.BaseElement.Parent.[As](Of HtmlTableRow)()
         Dim removeBtn As HtmlInputButton = row.Cells(6).Find.ByExpression(Of HtmlInputButton)("value=Remove")
         removeBtn.Click()
         'ad.WaitUntilHandled(2000)
         'Remove the dialog from the monitor
         Manager.DialogMonitor.RemoveDialog(ad)
    End If
    The code that is used in the script is displayed above. I noticed that the script runs fine in most of the cases but when there are multiple iterations that have to run with atleast one Data("FormAction") = Modify and have to execute a subtest, I am getting an "object reference not set to an instance of an object"
  18. Cody
    Admin
    Cody avatar
    3354 posts

    Posted 06 Dec 2012 Link to this post

    Hello,

    Before I can begin to diagnose this problem I need to know which line exactly is throwing the "object reference not set to an instance of an object" error message. When you get this error, it should contain a line number and a filename in the stack trace that goes along with it. For example:

    System.Exception: deliberate failure
       at VBTestStudioProject.WebAiiTest1.WebAiiTest1_CodedStep2() in C:\Users\gibson\Documents\Visual Studio 2010\Projects\VBTestStudioProject\VBTestStudioProject\WebAiiTest1.tstest.vb:line 85

    Please look in that file and pinpoint the line that is throwing this error. Then we can start diagnosing why that line of code is in error.

    Regards,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
  19. Mercedes
    Mercedes avatar
    6 posts
    Member since:
    Jul 2012

    Posted 07 Dec 2012 Link to this post

    Log.WriteLine(formcell.ToString())

    is the location where the error is occuring. It looks like the formcell doesn't return any value in that particular iteration. The error doesn't  occur for every iteration. If you look at the code i gave earlier, the error occurs only when the    
           Me.ExecuteTest("User Access\DeleteFormSchedule.tstest")

    step is executed. "DeleteFormSchedule.tstest" has an if condition which causes it to execute only in some cases. So after it is executed, and the next iteration is being executed, the error occurs.

    Please let me know if you need more information.
  20. Cody
    Admin
    Cody avatar
    3354 posts

    Posted 10 Dec 2012 Link to this post

    Hello Brian,

    Agreed it means that the line before it doesn't find anything:
    Dim formcell As HtmlTableCell = ActiveBrowser.Frames(0).Find.ByExpression(Of HtmlTableCell)("InnerText=" + Data("FormNumber"))

    Try adding this in from of that line:
    ActiveBrowser.RefreshDomTree()

    If that doesn't help let's look at this together via another GoToMeeting.

    All the best,
    Cody
    the Telerik team
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
Back to Top