DataDriven based testing best practices

[Mercedes]

Hi,
    I have a  test studio script that is data driven by a SQL server database. The web application against which the script was recorded is a policy system. The script works fine for new business system but i would like to know the best process to follow in creating a script that can be used for policy changes.

For example, if i have a set of entries on the new policy, the system would create a new policy but when there is only 1 entry change in the above set of entries, i do not want the complete script to execute but just the field that has been changed. The reason for that is, i have several hundred fields of data entry in the application and only one value could be changed.

Thank you

[Cody]

Hi Brian,

I apologize for the delay getting back to you on this.

What sort of policy are we talking about here? An insurance policy? A business policy? An HR policy? An advertisement policy?

Let me reiterate the problem just to make sure I understand it correctly:
1) When you create a policy there are several hundred fields that must be filled in, then the policy is saved to the system
2) You want to test making changes to an already existing policy stored in the system
3) You want to test changing just a few (or one) fields and do not want to have to re-enter all of the fields

Do I understand correctly?

Part of the problem here is how do you identify which field needs to be changed i.e. how can the test detect that field 2 or field 202 needs to be modified? Is there something in your data source that will specify this? Another detail to work out that is part of the same question is whether or not there are multiple pages involved e.g. fields 1-50 are on page 1, fields 51-99 are on page 2, so if somehow the data tells the test to set field 79 to X we need to know we have to navigate to page 2 in order to get to field 79.

Let's start by defining what the input data will look like when changing a policy. Will this input always include all data for all input fields, or will it somehow try to tell the test "change field n to X"? The test implementation varies greatly based on this answer. We'll go to the next step once you answer this starting question.

Kind regards,
Cody
the Telerik team

[Mercedes]

The policy i am talking about here is an insurance policy. What you specified is right. I have the input data for the new policy setup but for the changes i have not decided on how to do it as there are several fields on several pages. I needed to know what the process is to compare two records in a database so that i can run the script based on that.
If there is no way to compare two records on the database, i think it would be better to have all the fields in the record included and modify just the field that needs to be changed. That way the script will run through each step and pick the same values again and modify just the field that has changed. This would take longer time for the script to run which is the reason i wanted to find out better options. Please let me know if you have any questions.

[Cody]

Hello Brian,

I needed to know what the process is to compare two records in a database...

Test Studio does not have any built-in functionality for direct DB access or manipulation. We do have this code sample demonstrating how you can write your own code to access a DB. It would be possible (and not all that hard) to modify this code sample to compare two records in a DB and return the differences. You just have to come up with a SQL query to do so. Or you could perform two SQL queries to get two records in memory and use code to compare the two records.

Now the part I don't understand is what two DB's or records are you going to compare to get the differences? It's easy to see you have the main DB where the new policy has been created and saved. But what exactly will you compare against?

Greetings,
Cody
the Telerik team

[Mercedes]

Hello Cody,
      Thanks for taking the time to provide a solution. We have decided to run through the complete script and let test studio pick the values even if the record hasn't changed. This could cause the script to run longer than manually performing the actions but it would help us perform the work unattended.
       An unrelated question regarding my script. I might have to provide you a demo to properly explain this but let me try. I have an OK button on a web application, which when clicked gives me a pop-up. So i have a step in the script that clicks the OK button and then performs a dialog handle but when i am running the script, i noticed that the script executes the "click the OK button" but for some reason keeps searching for the button even after it has been executed. Can you please suggest why this is happening if you have seen similar issue before or i could provide you any information you need?

Thanks

[Cody]

Hi Brian,

Can you send me a screen shot of this dialog, or even better a video of this happening? Jing is a good free screen recorder. My one suspicion is that this dialog is a modal dialog. Modal dialogs have a bad habit of not returning control back to the test script until the modal dialog is closed. For this reason we normally set SimulateRealUser to true on the click step that causes the modal to open. This prevents the test from hanging indefinitely inside the click step.

All the best,
Cody
the Telerik team

[Mercedes]

Hi Cody,
    I recorded a video while playing back the script. I inserted a breakpoint, so that it will be easier to analyze the steps in a step-by-step fashion. There is a submit button which when clicked should display the pop-up that states that "it is an out of sequence transaction", and then the confirm dialog should click on ok. In the recording, you can see that the submit button by itself is clicking on the submit button and the pop-up OK button also and then the next step which is the actual confirm dialog is waiting for the dialog to appear. Please let me know if you have any questions.
         I did not see the option to set SimulateRealUser under the properties for either of the steps. The only option remotely close was to set SImulateRealClick to true which i tried unsuccessfully.


Thank you

[Cody]

Hi Brian,

My apologies, it's SimulateRealClick as shown in the attached screen shot (not SimulateRealUser... however that property does exist for the Set Text test steps).

I watched your video (had to find and download the correct CAMTASIO codec to play it) and am a bit concerned by what I see. Here's what I'm observing in the middle:

1) The test clicks the OK button for the Client Access Endorsement screen
2) This causes the JavaScript Confirm dialog to show up
3) Then this dialog goes away. I am concerned what caused this dialog to close. I find it hard to believe that Test Studio closed it simply because the test step still running was the one to click the OK button for the Client Access Endorsement screen.
4) Then the app proceeds to the next screen but now Test Studio is trying to handle the Confirm dialog that's already come and gone. It's out of sequence.

I'd like to look at this problem on your computer via GoToMeeting. Let me know what time zone you are in and your availability and I'll setup the meeting. I'm in Austin, TX which is Central Time (GMT -6).

All the best,
Cody
the Telerik team

[Mercedes]

I am in Eastern Time, Indiana. I am available from 1PM - 4 PM. Please let me know the time, so that i can share my test with you.

[Cody]

Hi,

All right. Let's target 2pm Central (3pm Eastern):

1.  Please join my meeting, Friday, November 16, 2012 at 2:00 PM Central Standard Time.
https://www2.gotomeeting.com/join/688837290

2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

Dial [phone]
Access Code: 688-837-290
Audio PIN: Shown after joining the meeting

Meeting ID: 688-837-290

GoToMeeting®
Online Meetings Made Easy™

Not at your computer? Click the link to join this meeting from your iPhone®, iPad® or Android® device via the GoToMeeting app.

All the best,
Cody
the Telerik team

[Cody]

Hi Nithin,

Just to document the outcome of our meeting:

1) We discovered you're trying to do conditional dialog handling based on the contents of the data source. You were conditionally calling a subtest. This approach doesn't work. The problem is that the dialog handler must be initialized and watching for the dialog before the dialog appears. Using the approach you had the dialog would appear, then the sub test would get control and initialize the dialog handler. By then it's too late since the dialog is already open. The subtest will ignore already open dialogs.

2) You had multiple Handle Confirm dialog handlers in your test. There is a known limitation in Test Studio... you can have only 1 dialog handler of a specific type in a test. You can get around this by using sub-tests to handle the additional dialogs of the same type as needed. We fixed your test today by splitting out the affected steps into sub-tests

3) We also created a coded step that will find a policy row based on the policy number, then find the machine Modify button and click on it.

Regards,
Cody
the Telerik team

[Mercedes]

Hi Cody,
       I tried using the steps that you created for me to find a policy row based on the policy number, then find the modify button and click on it. I had to make some changes and am getting the error "Index was out of range. Must be non-negative and less than the size of the collection."  I tried looking to see why the error was generating but could not figure it out. Any help is appreciated.

Thanks

[Cody]

Hi Brian,

It's impossible to say what is causing this without seeing the code involved and being able to run it and study the behavior on my machine. How about we look at this together in another GoToMeeting? On Friday I have only my 10:30 slot open. On Monday I can do 1 hour anywhere from 10:30 - Noon (Central time).

All the best,
Cody
the Telerik team

[Mercedes]

Cody,
      Friday at 10:30 am Central Time should be fine with me.

Thank you,
Nithin

[Cody]

Hi Nithin,

You got it:

1.  Please join my meeting, Friday, November 30, 2012 at 10:30 AM Central Standard Time.
https://www2.gotomeeting.com/join/452673762

2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

Dial [phone]
Access Code: 452-673-762
Audio PIN: Shown after joining the meeting

Meeting ID: 452-673-762

GoToMeeting®
Online Meetings Made Easy™

Not at your computer? Click the link to join this meeting from your iPhone®, iPad® or Android® device via the GoToMeeting app.

Greetings,
Cody
the Telerik team

[Cody]

Hi Nithin,

To summarize our meeting today:
1) We determined the wrong code was being used in the test having the problem. You will fix this yourself.
2) We looked at how to handle a case where a search returns "no client found" message. The conclusion was that you need to click on the first "select" button whether or not a client is found.
3) I showed you how to verify that text is set to Read-Only.

I am now closing this ticket as the main issue is resolved.

All the best,
Cody
the Telerik team

[Mercedes]

Log.WriteLine("Form Action is: " + Data("FormAction"))
If Data("FormAction") = "Add" Then
    Me.ExecuteTest("User Access\FormAdd.tstest")
    'Remove the dialog from the monitor
    Manager.DialogMonitor.RemoveDialog(ad)
Else If Data("FormAction") = "Modify"
     Dim formcell As HtmlTableCell = ActiveBrowser.Frames(0).Find.ByExpression(Of HtmlTableCell)("InnerText=" + Data("FormNumber"))
     Log.WriteLine(formcell.ToString())
     'Dim formcell As HtmlTableCell = ActiveBrowser.Frames(0).Find.ByExpression(Of HtmlTableCell)("InnerText=" + "PL 0002 0909")
     Dim row As HtmlTableRow = formcell.BaseElement.Parent.[As](Of HtmlTableRow)()
     Dim modifyBtn As HtmlInputButton = row.Cells(6).Find.ByExpression(Of HtmlInputButton)("value=Modify")
     modifyBtn.Click()
     'Remove the dialog from the monitor
     Manager.DialogMonitor.RemoveDialog(ad)
     Me.ExecuteTest("User Access\DeleteFormSchedule.tstest")
     Me.ExecuteTest("User Access\FormDetail.tstest")
Else If Data("FormAction") = "Remove" Then
     Dim formcell As HtmlTableCell = ActiveBrowser.Frames(0).Find.ByExpression(Of HtmlTableCell)("InnerText=" + Data("FormNumber"))
     Log.WriteLine(formcell.ToString())
     Dim row As HtmlTableRow = formcell.BaseElement.Parent.[As](Of HtmlTableRow)()
     Dim removeBtn As HtmlInputButton = row.Cells(6).Find.ByExpression(Of HtmlInputButton)("value=Remove")
     removeBtn.Click()
     'ad.WaitUntilHandled(2000)
     'Remove the dialog from the monitor
     Manager.DialogMonitor.RemoveDialog(ad)
End If

The code that is used in the script is displayed above. I noticed that the script runs fine in most of the cases but when there are multiple iterations that have to run with atleast one Data("FormAction") = Modify and have to execute a subtest, I am getting an "object reference not set to an instance of an object"

[Cody]

Hello,

Before I can begin to diagnose this problem I need to know which line exactly is throwing the "object reference not set to an instance of an object" error message. When you get this error, it should contain a line number and a filename in the stack trace that goes along with it. For example:

System.Exception: deliberate failure
   at VBTestStudioProject.WebAiiTest1.WebAiiTest1_CodedStep2() in C:\Users\gibson\Documents\Visual Studio 2010\Projects\VBTestStudioProject\VBTestStudioProject\WebAiiTest1.tstest.vb:line 85

Please look in that file and pinpoint the line that is throwing this error. Then we can start diagnosing why that line of code is in error.

Regards,
Cody
the Telerik team

[Mercedes]

Log.WriteLine(formcell.ToString())

is the location where the error is occuring. It looks like the formcell doesn't return any value in that particular iteration. The error doesn't  occur for every iteration. If you look at the code i gave earlier, the error occurs only when the    
       Me.ExecuteTest("User Access\DeleteFormSchedule.tstest")
step is executed. "DeleteFormSchedule.tstest" has an if condition which causes it to execute only in some cases. So after it is executed, and the next iteration is being executed, the error occurs.

Please let me know if you need more information.

[Cody]

Hello Brian,

Agreed it means that the line before it doesn't find anything:
Dim formcell As HtmlTableCell = ActiveBrowser.Frames(0).Find.ByExpression(Of HtmlTableCell)("InnerText=" + Data("FormNumber"))

Try adding this in from of that line:
ActiveBrowser.RefreshDomTree()

If that doesn't help let's look at this together via another GoToMeeting.

All the best,
Cody
the Telerik team