Tests fail: Unable to find assembly ArtOfTest.WebAiiVSIP

[Nuno]

Hi all,

I installed WebUI 2010.2 Dev Edition to create UI tests for Silverlight Apps.
I'm also using TFS 2010 Continuously Integration. When I queue a new build (tests are set to run in a web server with an active Test Agent) I get this error as test result (see attached image).

Even when I create a complete new Test Project with just a MS coded UI test (No Telerik WebAii test in it) I get the same error.

Any ideas why?

Thank you,
Nuno

[Cody]

Hi Nuno,

The test agent computer does not have WebUI Test Studio installed. This is required for tests to run in a build server/test agent environment.

During your evaluation period you can install the Dev Edition on your test agent (assuming it has Visual Studio installed on it). When you are ready to purchase, you can uninstall that, obtain a license to our Run Time Edition (only available for purchase and retails for just $199) and install that instead on your test agent computer.

All the best,
Cody
the Telerik team

[Nuno]

Hi Cody,

Thanks for your reply. 
Ok, I installed WebUI Test Studio Dev Edidion on the test agent computer.

Now, when I run a test on the test agent computer it opens a new browser window but nothing happens. The window just keeps blank all time until the test fails due to a timeout exception (please see attached image). It never navigates to the target URL as it was supposed to.

I changed the Execution command timeout to 60000 but it still fails.

If I run the same test using the local test settings it runs perfectly on my machine.

Any ideas?

Regards,
Nuno

[Cody]

Hello Nuno,

Let's determine if it's a problem running any test on the test agent or just trying to run the test via a build. On the test agent computer, can you manually launch Visual Studio, load and run the test? Does this work? If so it proves the installation and setup of WebUI on that machine is fine and the problem is with running tests in the test agent environment.

Are you running the test agent as a service? It probably needs to run in "interactive" mode as we discuss in our documentation and on Microsofts website.

Kind regards,
Cody
the Telerik team

[Nuno]

Hello Cody,

I'm able to start (manually/from a build) and successfully run standard Microsoft Coded UI tests on the test agent machine, either locally or triggered from another machine (Build agent machine). 

When it comes to run a WebAii test on that same machine it fails when running locally and also remotely. As I mentioned before, just a blank IE window pops up and remain like that untill the test fails. It seems the test agent can't interact with that window.

I'm running the test agent as a service in "interactive" mode since it is mandatory to interact with the UI.

On the test agent machine I installed the same software I installed on my local machine. VS2010 and WebUI Test Studio Dev Edition 2010.02.
It seems there is something wrong with the WebUI Test Studio installation... don't know what exactly.

Any ideas at this point?

Thank you,
Nuno

[Nuno]

Hi again Cody,

Just an update, find attached a detailed list of the tested scenarios and respective test results.

Hope this helps to track the problem.

Thanks for your help.

Regards,
Nuno

[Cody]

Hello Nuno,

If I understand correctly this sequence does not work:

On the test agent computer you login locallly
You start Visual Studio
You load a test project
You try to QuickExecute any WebAii test

The result is the IE window opens and nothing more. Yet the exact same test runs just fine when you run it locally on your development computer?

If that is all true, I'd like to focus on getting WebAii tests to run within the Visual Studio IDE (i.e. Quick Execute) while logged into your test agent computer locally.

Is IE7 being used on this computer? We require IE7 or above. If it's IE7, is the IE Developer Toolbar installed? We know of an incompatibility with that toolbar and our software. Can you disable or uninstall this add-on? If that doesn't help can you disable all other IE add-ons except ours please? There may be some other add-on that is causing a conflict. Our add-ons are "WebUI TestStudio Recorder" and "WebAii Test Execution Extension"

Best wishes,
Cody
the Telerik team

[Nuno]

Hi Cody,

Thanks for your reply and sorry for my delay... I've been pushed to others running projects since last week...
At present moment all tests run successfully when QuickExecuted from my VS2010 IDE on my local machine. (local.testsettings)
Success as well when QuickExecuted from VS2010 IDE installed on the test agent machine. (local.testsettings)

The problem remains on the Build machine, WebAii test fail when QuickExecuted from my VS2010 IDE using local.testsettings or remote.testsettings. (this last one pointing to the test agent machine)

The IE version on the Build machine is: 8.0.6001.18702 and on the Test Agent machine is: 8.0.6001.18943.

Please, find attached print screens of the Add-ons windows as when the tests where QuickExecuted from my VS2010 IDE.

Thanks for your help Cody.

Regards,
Nuno Senica

[Nikolai]

Hello Nuno,

Just to make sure that the build machine is properly set. Can you please check:
1. Does this file exists:  
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies\ArtOfTest.WebAiiVSIP.dll if it does not please reinstall the Telerik WebUI TestStudio.
2. Is the running OS Server edition(2003/2008) if it is there are some special configurations you will need to set. You can find them here.
3. Here you find also how to configure your browser if you have not done this already.

Greetings,
Nikolai
the Telerik team

[Nuno]

Hi Nikolai and Cody,

Thanks for your help. I followed those links steps and I'm now being able to successfully run a test on the remote test machine.

Now I need to run Telerik Webaii tests as part of my TFS 2010 automated builds.
Is it necessary to convert all steps to code (MS Unit Test) after recorded the test from a browser window (pointing to a Silverlight App) ?? If yes, does that means every time a tester needs to record more steps he will have to re-work on the "aii" file and that re-convert all steps to code? (I'm asking this because the tester has no coding experience).

Also, can I have two solutions, one with the Silverlight App projects and another with the separated test projects? As opposed to one single solution with Silverlight and Test projects together. If yes, can you please explain to me how should I configure the Build definition?

Once again, thank you for your kind support.

Nuno Senica

[Keaegan]

Hi Nuno,

Those are some good questions. I'll start with your first one:

Is it necessary to convert steps to code each time you record new steps with WebUI Test Studio Dev. Edition?
No, converting to code is not necessary unless your specific needs require it due to special setup or a known issue (the most likely example of this is a modal window interaction needing a .click() event instead of a .mouseclick() event in order to be handled correctly, which would need to be hand-coded).

The agent in question should be able to record steps, and provide the entire test project file to you (so that you can convert it to code if you need to), but the GUI itself will allow the steps to be created without needing to be "converted to code" each time.

Can I have to separate solutions(projects)?
Yes you can! You can accomplish what you are wanting to do one of two ways:
Create a separate Project/Solution for Silverlight and another one for your other, non-silverlight tests (this is the preferred method)

Alternatively, You can create separate Test Projects within the same Solution. This method is not suggested.

If you still need assistance with configuring the Build Definition, please let me know. If necessary, we can assist with the creation of it.

Greetings,
Keaegan
the Telerik team

[Nuno]

Hi Keaegan,

When I asked if it was necessary to convert all steps to code (MS Unit Test) after recorded I also asked because I'm not being able to run a WebAaii test type as part of my build. On the MSBuild log says: "No tests to execute."
If I convert all steps to code I'll create a new test file type of MS Unit Test. This way my automated build will find/run the test.
What am I missing here?

Another question, my WebAii test is associated with a Work Item. So, after both the build and test successfully finish, where can I find the WebAii test results details on TFS 2010 ?
I can tell you tests are being published into TFS because I get this lines on the build log:

"Publishing results of test run...

..Publish completed successfully."

Thanks for your help.

Regards,
Nuno Senica

[Cody]

Hi Nuno,

I am sorry for the confusion. I am attaching a document that describes how to make .aii tests run in a TFS 2010 build. The other way of making .aii tests run in TFS builds is to manually add MSTest commands to your TFS build script. We have a video on how to run .aii tests from the command line using Microsoft MSTest command line tool.

Greetings,
Cody

the Telerik team

[Nuno]

Hi Cody,

That did the trick! Thank you!

Now, regarding testing, I'm not being able to create aii tests within a Silverlight Unit Test Application (project). Shouldn't this be possible?
(I'd like to have the mentioned project to contain aii and unit tests for my SL app)

Thank you for the great support!

Regards,
Nuno Senica

[Cody]

Hi Nuno,

I am glad we are making progress on your automation problems.

What sort of project are you selecting in Visual Studio in order to create your Silverlight Unit Test's in? It is true that WebAii tests can only be added to Visual Studio Test Projects (either C# or VB) as shown in the attached screenshot. I don't know what you are creating your unit tests in, but I would think you would want them in a test project as well. You can mix and match all sorts of test types within these Visual Studio test projects.

Greetings,
Cody

the Telerik team