This question is locked. New answers and comments are not allowed.
Stephen Gyves
Top achievements
Rank 1
Rank 1
Stephen Gyves
asked on 29 Dec 2009, 12:17 AM
Okay, I know the first thing you are thinking, this is not a function of the design canvas. Well that is precisely my problem, so I have to open up something for discussion. Firstly, let me explain my ideal situation:
All controls have an Automation ID by which I can reference the control and access all pertinent information regarding the control and fluidly execute functions of the control. Apparently, this would be more of a manual approach to the testing.
The current situation with using the WebUI Studio product is that it assumes many things. It assumes that data in drop downs (as well as sort order, and in some cases, order of controls on the page) will not change. I recently found this to be the case regarding trying to drive the combo box selection by a data source (not allowed) and or identify a selection by name somehow rather than it's index. There appeared to be a setting to identify combo box selection by text but when I looked in the Elements Explorer after recording with this mode enabled, it was still being found by its index.
And now for some questions; Am I aiming for too lofty of a goal to assume that we will be able to create some framework that will enable us to make reasonably resilient tests that will not break when data changes? Is it possible to drive a combo box with data either through manual code without knowing the index of the list item and only the text? I would like to not have to make another test in order to change up combo box selection changes. I hope it doesn't seem like I am trying to knock the testing framework, I just really need to get a hold on some sort of reality that tells me where I am going to get the best return on investment for our automation efforts. It has been readily evident that our team is trying to use the framework in ways that have not been demanded by others, and that leads me to question my approach. Are we automating too early? I can honestly tell you that at the current stage of our project, I have little faith that the tests recorded will work for long if we depend on the assumptions earlier mentioned. If you could please offer your opinions and suggestions I would greatly appreciate it. Thank you.
All controls have an Automation ID by which I can reference the control and access all pertinent information regarding the control and fluidly execute functions of the control. Apparently, this would be more of a manual approach to the testing.
The current situation with using the WebUI Studio product is that it assumes many things. It assumes that data in drop downs (as well as sort order, and in some cases, order of controls on the page) will not change. I recently found this to be the case regarding trying to drive the combo box selection by a data source (not allowed) and or identify a selection by name somehow rather than it's index. There appeared to be a setting to identify combo box selection by text but when I looked in the Elements Explorer after recording with this mode enabled, it was still being found by its index.
And now for some questions; Am I aiming for too lofty of a goal to assume that we will be able to create some framework that will enable us to make reasonably resilient tests that will not break when data changes? Is it possible to drive a combo box with data either through manual code without knowing the index of the list item and only the text? I would like to not have to make another test in order to change up combo box selection changes. I hope it doesn't seem like I am trying to knock the testing framework, I just really need to get a hold on some sort of reality that tells me where I am going to get the best return on investment for our automation efforts. It has been readily evident that our team is trying to use the framework in ways that have not been demanded by others, and that leads me to question my approach. Are we automating too early? I can honestly tell you that at the current stage of our project, I have little faith that the tests recorded will work for long if we depend on the assumptions earlier mentioned. If you could please offer your opinions and suggestions I would greatly appreciate it. Thank you.
2 Answers, 1 is accepted
0
Hi Stephen Gyves,
Let me start by addressing your question regarding using data to drive the selection of a ComboBox. First, you don't mention it so I am going to assume you're referring to a Silverlight ComboBox contained in a Silverlight application (since standard HTML does not have a ComboBox control).
One problem we have to take into consideration is that the ComboBoxItem's contained in a ComboBox may be something other than text. Silverlight allows you to use things like images or any other UI element as your ComboBoxItem's. As a result, the index is the only property common to all possible ComboBoxItem's.
Having said that, a list of text items may be the most common use of ComboBox's and in that scenario selecting by text would be a very natural way of selecting, and even preferred (as you point out) for test robustness because it makes the test immune to the index order or even the number of the ComboBoxItem's.
The good news is that there is a fairly simply method for implementing a "select by text" feature in a small code behind method. For demonstration purposes I used this web page containing ComboBox's http://helen.org.ua/caramel as my sample page.
Step 1 of my simple test navigates to the sample page.
Step 2 opens the ComboBox drop down by clicking on it.
Step 3 contains the real "magic". i implemented a code behind method to find and click on the right ComboBoxItem using a data driven source. The code look like this:
I will log a feature request to implement something like this in WebUI so that you don't have to resort to code behind methods in the future. Be aware the above code will break (throw an exception) if the cboxItem.Content is anything but a string.
Now let's talk about your ideal situation. Yes, it would be absolutely wonderful if Silverlight programmers would add Automation ID's to every control and be sure they are unique. In practice it is often difficult to get software developers to do much of anything strictly for test support. I've even heard of a few managers that instruct their developers to focus just on functionality and robustness. Naturally testing support greatly suffers in such an environment.
Another complication is code reuse and third party custom controls. Even if the Automation ID's are unique within a single instance of a custom control, imagine the havoc that can happen if multiple instances of the same custom control are placed on the same web page or in the same Silverlight application.
We realize this is the reality of the situation. That's why WebUI does it's very best to automatically generate Find Expressions for locating elements on a web page or in a Silverlight application to act on that are as robust as possible despite web pages and Siverlight applications changing while being developed (or even after launch and fixes/changes are applied). We try to do the best we can given an imperfect testing environment.
Our end goal is the same, to create very robust test automation. We are trying to accomplish that very thing with WebUI without relying on any special support from developers. It is a difficult task to create 100% robust test automation under that condition. There are a lot of variables that we simply don't have control over. Therefore we do everything we can to make it as robust as possible while, at the same time, making maintenance of your tests as easy and painless as possible as well. Then when a test does break (due to a breaking change by the developer) it doesn't take you days to fix your test automation.
I hope I have addressed your concerns as well as given you additional tools (at least for ComboBox's) for making your tests more robust.
Sincerely yours,
Cody
the Telerik team
Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.
Let me start by addressing your question regarding using data to drive the selection of a ComboBox. First, you don't mention it so I am going to assume you're referring to a Silverlight ComboBox contained in a Silverlight application (since standard HTML does not have a ComboBox control).
One problem we have to take into consideration is that the ComboBoxItem's contained in a ComboBox may be something other than text. Silverlight allows you to use things like images or any other UI element as your ComboBoxItem's. As a result, the index is the only property common to all possible ComboBoxItem's.
Having said that, a list of text items may be the most common use of ComboBox's and in that scenario selecting by text would be a very natural way of selecting, and even preferred (as you point out) for test robustness because it makes the test immune to the index order or even the number of the ComboBoxItem's.
The good news is that there is a fairly simply method for implementing a "select by text" feature in a small code behind method. For demonstration purposes I used this web page containing ComboBox's http://helen.org.ua/caramel as my sample page.
Step 1 of my simple test navigates to the sample page.
Step 2 opens the ComboBox drop down by clicking on it.
Step 3 contains the real "magic". i implemented a code behind method to find and click on the right ComboBoxItem using a data driven source. The code look like this:
[CodedStep(@"Click on data driven ComboBoxItem", RequiresSilverlight = true)] public void SweetControls_CodedStep() { // Scan through the list of ComboBoxItem's looking for the right one to click on, by text foreach (ComboBoxItem cboxItem in Pages.Pale2.SilverlightApp.Item0Combobox.Items) { if (cboxItem.Content as string == (string)Data[0]) { // We found a match. Now click on it and break out of the loop. cboxItem.User.Click();
break;
} } // TODO: Add logic to detect when we failed to find the right item and fail the test } I will log a feature request to implement something like this in WebUI so that you don't have to resort to code behind methods in the future. Be aware the above code will break (throw an exception) if the cboxItem.Content is anything but a string.
Now let's talk about your ideal situation. Yes, it would be absolutely wonderful if Silverlight programmers would add Automation ID's to every control and be sure they are unique. In practice it is often difficult to get software developers to do much of anything strictly for test support. I've even heard of a few managers that instruct their developers to focus just on functionality and robustness. Naturally testing support greatly suffers in such an environment.
Another complication is code reuse and third party custom controls. Even if the Automation ID's are unique within a single instance of a custom control, imagine the havoc that can happen if multiple instances of the same custom control are placed on the same web page or in the same Silverlight application.
We realize this is the reality of the situation. That's why WebUI does it's very best to automatically generate Find Expressions for locating elements on a web page or in a Silverlight application to act on that are as robust as possible despite web pages and Siverlight applications changing while being developed (or even after launch and fixes/changes are applied). We try to do the best we can given an imperfect testing environment.
Our end goal is the same, to create very robust test automation. We are trying to accomplish that very thing with WebUI without relying on any special support from developers. It is a difficult task to create 100% robust test automation under that condition. There are a lot of variables that we simply don't have control over. Therefore we do everything we can to make it as robust as possible while, at the same time, making maintenance of your tests as easy and painless as possible as well. Then when a test does break (due to a breaking change by the developer) it doesn't take you days to fix your test automation.
I hope I have addressed your concerns as well as given you additional tools (at least for ComboBox's) for making your tests more robust.
Sincerely yours,
Cody
the Telerik team
Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.
0
Stephen Gyves
Top achievements
Rank 1
Rank 1
answered on 29 Dec 2009, 05:53 PM
Thank you for your detailed reply to my issue. I got a kick out of reading your response to my 'ideal situation'. Of course we all know that doesn't exisist. I also never had any doubt as to the goal of your automation framework to make it as flexible as possible. In addition, your teams willingness to work with QA developers to augment the existing functionality is admirable.
I had also came up with a similar method to handle the text look up in a ComboBox. Thank you for your code sample as well.
I also have found that often composite controls are reused throughout our Silverlight application thus repeating ID's. Yes, total buy-in from the development to carefully implement Automation Id's is going to be spotty at best. However, you gave me your analysis of how to approach this dynamic and that is all I really wanted. Hopefully we can continue this working relationship of continuous improvement in Silverlight test automation. Your feature requests are greatly appreciated and we look forward to future builds of the test framework as well. Thank you.