UI Automation and MSAA (Microsoft Active Accessibility) in UI for Silverlight General Discussions

Vincent

20 posts

Member since:
Oct 2008

Posted 09 Oct 2008 Link to this post

I am the supervisor for the Quality Assurance team for our company and during the course of writing integration/automated test sfor our new Silverlight application I noticed that there were certain controls I could not interact with using UI automation. These were the RadWindow and RadSlider. All our other controls, currently, are Microsoft controls and we have had no problems interacting with them.

We use a tool called UISpy to gage the accessibility of an application. UISpy is a tool that enables developers and testers to view and interact with the user interface elements of an application. As such, it can identify elements that are visible to assistive technologies (like screen readers) for the blind, and UI automation tools for automated testing. From what we have been able to gather Telerik’s Silverlight controls are not visible to assistive technologies.

We are not sure if Telerik is aware of this or if there are future plans to support assistive technologies. Currently, this is an issue for us and the adaptation of Telerik’s RadControls.

Any insight on this matter would be greatly appreciated.

Regards,
Vincent

Valeri Hristov

Admin

2252 posts

Posted 09 Oct 2008 Link to this post

Hi Vincent,

Indeed, currently only very few of RadControls for Silverlight have UI Automation functionality. We are aware of that and we are going to provide support for UI Automation in virtually all of the controls in the upcoming official release of RadControls for Silverlight this November.

Could you be so kind to share what tools for automated testing are you using? I am very curious, because we are searching for a good tool to test our own controls and applications.

Best wishes,
Valeri Hristov
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

Vincent

20 posts

Member since:
Oct 2008

Posted 09 Oct 2008 Link to this post

Thank you for your quick response.

We're currently using the White Framework for Unit Testing and we're using Test Complete as a tool with a full testing integration framework.
Automated QA's Test Complete has a MSAA add-on that exposes some controls, however it is not as robust as UISpy. What are you using?

If you need beta testing for your controls while you are making them more compliant we'd be interested in participating.

Konstantin Petkov

Admin

1911 posts

Posted 10 Oct 2008 Link to this post

Hello Vincent,

Thank you for your willingness to participate and for your answer. Indeed, WPF White tool is definitely a good option for UI Automation testing and we have already started using that too. Fortunately, its creator recently confirmed he is working on Silverlight support as well, so we will most probably see that included in the next 0.18 version of White tool.

I'm happy to inform you part of the controls UI Automation support is already available. I'd suggest you download the next official update as soon as we announce it. We just scheduled that for Tuesday next week. We've already worked on RadSlider UI Automation support which I hope you'll enjoy.

Your feedback on this topic is greatly appreciated so keep us notified on any issues/requests.

Best wishes,
Konstantin Petkov
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

Konstantin Petkov

Admin

1911 posts

Posted 20 Oct 2008 Link to this post

Hello Vincent,

Were you able to check the RadSlider UI Automation support in RadControls for Silverlight 2 RC1 we released last week?

We were discussing here which controls to focus on next on UI Automation implementation and I remembered your request about RadWindow. We've put that with higher priority.

Are you interested in any other controls UI Automation support? Feel free to post requests here so that we can prioritize the tasks immediately, thank you!

Regards,
Konstantin Petkov
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

Vincent

20 posts

Member since:
Oct 2008

Posted 18 Dec 2008 Link to this post

Hello Valeri,

We're still having the same issue with the RadWindow. Has there been any new releases since, and if so has this been addressed?

Thank you,
Vincent

Konstantin Petkov

Admin

1911 posts

Posted 19 Dec 2008 Link to this post

Hello Vincent,

The Window control UI automation support has been provided since the official release of RadControls for Silverlight in the beginning of November:

http://blogs.telerik.com/konstantinpetkov/posts/08-11-07/UI_Automation_Support_in_RadControls_for_Silverlight_Q3_2008.aspx

I just checked out the latest Window online (we just release Service Pack 2 of Q3 2008) and it is accessible through UI Spy. Here is a link to the live demos for your convenience:

http://demos.telerik.com/silverlight/#Window/FirstLook

Can you please take a look at this and let us know if you still cannot access the Window control even with the latest version? If the issue you hit is more specific, can you please elaborate more on that so that we can investigate it and get back to you with our findings/resolution. Thank you!

All the best,
Konstantin Petkov
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

Vincent

20 posts

Member since:
Oct 2008

Posted 22 Dec 2008 Link to this post

Hi Konstantin,

Thank you for the reply. We checked the control using UISpy and it is identifying it as you mentioned above. However, we are also using Automated QA's Test Complete MSAA add-on. This exposes objects, methods and properties using the IAccessible interface.

Here's a little more information regarding MSAA with Test Complete (in this example they use it on Access):
http://www.automatedqa.com/community/Home/Articles/tabid/38/articleType/ArticleView/articleId/404/Default.aspx

Do you have any information that may help us?

Thank you,
Vincent

Konstantin Petkov

Admin

1911 posts

Posted 23 Dec 2008 Link to this post

Hello Vincent,

I'm afraid we have to admit about the missing MSAA support in RadWindow for Silverlight at the time present. We have focused on UI Automation since the UI Automation is intended to be the successor to Microsoft Active Accessibility.

We will research the requirements to support MSAA in RadWindow and implement that for a future version of the product.

Thank you for the feedback, Vincent, and please, excuse us for any inconvenience this has caused. Let us know if there is something else that we could help with.

Greetings,
Konstantin Petkov
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

Vincent

20 posts

Member since:
Oct 2008

Posted 30 Dec 2008 Link to this post

Thank you very much for your prompt replies.

Are you guys doing any automated testing on the Silverlight controls? If so, what are you using?

Also, do you have any idea about a timeframe regarding when we can expect new news about MSAA?

Thanks,
Vincent

Konstantin Petkov

Admin

1911 posts

Posted 31 Dec 2008 Link to this post

Hi Vincent,

First about the automated testing. As you have probably noticed, the third-party test automation solutions supporting Silverlight are still quite limited. Even the big players on the market like Automated QA's TestComplete is limited to the UI Automation and MSAA plug-ins. By limitation I mean the necessary implementation of UI Automation support of the Silverlight custom controls for example, which is a continuous task we'll be keep working on.

Some other tools you might want to look at are the UIA Verify library available on CodePlex as well as the ArtOfTest's WebAii for Silverlight. I've already blogged twice about the UIA Verify and the posts here and here can give you some quick insight about the functionality this tool offers. As to the WebAii framework, they have excellent tutorial posted here.

Regarding the MSAA support in RadWindow. I've worked closely with the devs here on investigating that and our conclusions follow. The IAccessible interface itself is something we have no access to -- it is just a COM interface. The MSAA support should be covered by the UI Automation implementation itself. We suspect we still haven't completed the UI Automation of RadWindow for example which probably causes the issue you've faced. To be more specific, we will give the InvokePattern a try shortly and test with the MSAA plug-in in TestComplete. On a side note, the Narrator speach engine seems to be reacting upon RadWindow user interaction. I will surely update this thread as soon as we have news on that.

Sincerely yours,
Konstantin Petkov
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

Vincent

20 posts

Member since:
Oct 2008

Posted 28 Jan 2009 Link to this post

Hello,

I was wondering if there was an update with this issue? Is the Rad Window MSAA accessible now?

Thanks,
Vincent

Nikolay

Admin

3997 posts

Posted 30 Jan 2009 Link to this post

Hi Vincent,

No progress, yet. This task is in a research phase right now as it appeared to be more difficult than we initially thought. We will surely post an update in this forum as soon as we have certain results.

Kind regards,
Nick
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

Vincent

20 posts

Member since:
Oct 2008

Posted 10 Feb 2009 Link to this post

Thank you for your reply Nik! We understand there will be a new release coming sometime this month. Will MSAA for the Popup window make it into that release?

Thank you,
Vincent

Nikolay

Admin

3997 posts

Posted 12 Feb 2009 Link to this post

Hi Vincent,

Unfortunately, we will not manage to include MSAA for the Popup in the forthcoming release.

Greetings,
Nick
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

Vincent

20 posts

Member since:
Oct 2008

Posted 12 Mar 2009 Link to this post

Hello,

Our MSAA Object Browser was unable to detect any of the controls on this page: http://demos.telerik.com/silverlight

We were able to see objects using MSAA on previous demo pages. Is there a specific reason for this?

Thank you,
Vincent Lomanto

Hristo

Admin

832 posts

Posted 16 Mar 2009 Link to this post

Hello Vincent,

There is no specific reason for not seeing object in MSAA. This is probably due to some changes in our QSF application.
As soon as we find what is the cause for this I'll let you know.

Kind regards,
Hristo
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Check out the tips for optimizing your support resource searches.

Vincent

20 posts

Member since:
Oct 2008

Posted 30 Mar 2009 Link to this post

Hello,

Is there any new information regarding the issue of us not being able to see objects in MSAA on the demo page?

Thank you,
Vincent

Nikolay

Admin

3997 posts

Posted 02 Apr 2009 Link to this post

Hello Vincent,

No progress yet. We will continue our research and development on the matter though. We will post in this forum thread as soon as we have certain results.

All the best,
Nick
the Telerik team

Check out Telerik Trainer , the state of the art learning tool for Telerik products.

hwsoderlund

419 posts

Member since:
Aug 2006

Posted 23 Nov 2009 Link to this post

It's been a while since this thread was active, but I'll post the answer anyway. The reason that UISpy etc. cannot see anything on the telerik demo page is that the "windowless" parameter is set to true in the hosting html page. I don't know why, but running in windowless mode effectively kills all accessibility.

Milan

Admin

1989 posts

Posted 24 Nov 2009 Link to this post

Hi hwsoderlund,

Thank you for sharing this information on our forums. We really appreciate your efforts. I have updated your Telerik points.

All the best,
Milan
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.

hwsoderlund

419 posts

Member since:
Aug 2006

Posted 24 Nov 2009 Link to this post

No problem. One more question regarding this topic: When Silverlight 4 comes out, with native support for the right mouse button and context menus, will it still be necessary to run in windowless mode? Because at the moment I have to choose between being accessible and supporting the right mouse button, which is really annoying.

Valeri Hristov

Admin

2252 posts

Posted 27 Nov 2009 Link to this post

Hi Henrik,

We will update all controls, that rely on custom Framework extensions, such as the right click, to use the new standard Silverlight functionality. This would be beneficial for everyone - less sources for problems in our code, less code to maintain, smaller assemblies, will work in all supported by Silverlight scenarios, etc.

Best wishes,
Valeri Hristov
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.