Executing tests withing VS2010 against my agents fails

[Jamie]

I added a test settings file to my project, and believe I have set it up correctly however when I try to run tests from VS2010, I get the following error:

Error 9/20/2011 8:43:41 AM Failed to queue test run 'Jamie@API8406 2011-09-20 08:43:40': Unable to find assembly 'ArtOfTest.WebAiiVSIP, Version=2011.1.502.0, Culture=neutral, PublicKeyToken=14176981d1873d86'. API8406


Another associate of mine is able to run tests from his machine through VS2010 against the Agents successfully, but we cannot figure out what is different between our machines.  What am I missing to be able to run my tests against the controller/agent system?

What is ArtOfTest.WebAiiVSIP?

We are currently using the licensed version of v2.0.50727/2011.1.502.0.

[Plamen]

Hello Jamie,

I'm sorry you are experiencing this problem. ArtOfTest.WebAiiVSIP is actually our Visual Studio plug-in. That error is typically seen when the test agent computer does not have Test Studio installed, which is required for tests to run in a build server/test agent environment. Is this a remote agent? The reason for this error could also be if there is a version mismatch. You mentioned that the version you are currently using is 2011.1.502, but I saw from your account you also have downloaded 712 and 829. If it is a remote agent system please ensure its version of Test Studio is the same.   

Hope to here from you soon!

Greetings,
Plamen
the Telerik team

[Jamie]

Yes, all the agents are running the same version, 111.5.2.0.  I can run the tests from the batch file with compiled dlls just fine.  And my associate can run his tests from his workstation through VS2010 against the same agents.  So I do not think it is a version issue.

Our 3 test agents are running Telerik Test Execution 2011.1 v111.5.2.0
Out 1 test controller is running Telerik Test Execution 2011.1 v111.5.2.0

We are using the older version of the framework due to other issues I am experiencing with the newer versions.  Those issues are documented and I am waiting on resolution before I can upgrade.

I would like to be able to run exactly like my associate where I point my VS2010 project to the agents, however I cannot due to the issue in the first post.

[Jamie]

I looked at my associates machine and it turns out he has not installed the licensed Telerik Studio we just purchased.  Which might be why this works for him.  Either way I still would like to be able to run my tests through the controller, from Visual Studio.

[Jamie]

Still looking for advice on this?  Is the ArtOfTest.WebAiiVSIP a file I should have on my computer?

[Cody]

Hi Jamie,

I am sorry you continue to have this problem. As Plamen already explained ArtOfTest.WebAiiVSIP.dll is our Visual Studio plug-in. It is installed by our Test Studio installer at C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies for VS2010 or C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies for VS2008.

I wonder if this is a .NET framework version issue. Are you both using the same version of Visual Studio? VS2008 will force test projects to use .NET 3.5 while VS2010 will force tests to use .NET 4.0. Do your tests refer to any extra dll's other than Microsoft and Telerik's?

Kind regards,
Cody
the Telerik team

[Jamie]

As I stated in a previous post, my associate has not installed the licensed version of Test Studio yet.  I thought he had, but after further exploration I determined he has not installed the licensed version of Test Studio yet.  He ist still running just the free edition Testing Framework.

I have attached some print screens from VS2010 in hopes this information will help us come to a resolution.

I am running VS2010, and my project Target framework is .Net Framework 4

If you need me to try anything or need more information please let me know, thanks.

[Jamie]

I created a simple Test Studio Project in VS2010 with one test and I get the same error when trying to run my one test.  If I debug my test it works, but when I try to run it, using my TestSettings.testsettings file, to run against my controller, I get the "Unable to find assembly ArtOfTest.WebAiiVSIP" error.  I have attached my simple project.

I have found both versions of the WebAiiVSIP dlls on my computer, so I know they are there.

[Jamie]

My associate upgraded to the licensed version of Telerik Test Studio (v502) and now has the same issue as me.

[Cody]

Hello Jamie,

I'd like to look at this problem on your computer via GoToMeeting. Let me know what time zone you are in and your availability and I'll setup the meeting. I'm in Austin, TX which is Central Time (GMT -6).

Kind regards,
Cody
the Telerik team

[Jamie]

I am working with my IT department on the process for allowing GoToMeeting with 3rd party vendors.  Hopefully will have this resovled shortly.

In the meantime my fellow associate had these questions for you:

Where does the ArtOfTest.WebAiiVSIP.dll come from? The test studio or the runtime or both?

Where should the ArtOfTest.WebAiiVSIP.dll get placed when the test studio and/or runtime is installed?

Does this dll need to be on the controller and all agents and if so where exactly, or is this just a development dll?

Is this dll supposed to be referenced in our VS solution or setup as a deployment item?

[Jamie]

GoToMeeting approval has been granted.  I am in the Central Time as well (Milwaukee, WI). 

Unfortunately today and tomorrow are booked for me.

Oct 4 from 11:00 am - 3:30 pm
Oct 5 from 11:00 am - 3:30 pm
Oct 6 from 1:00 pm - 3:30 pm
Oct 7 from 11:00 am - 3:00 pm


If these dates and times do not work for you please let me know and I will update with more dates and times, thanks.

[Cody]

Hello Jamie,

ArtOfTest.WebAiiVSIP.dll is installed by all of our installers (Test Studio, Test Studio Express, Run-Time edition). it is always installed at "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies".

No you should not be including it as a reference to your test projects. It does not contain any functions that are callable/usable by a test. it is our Visual Studio plug-in. It provides:
- The menu items and other UI extensions you can see in Visual Studio
- The "Generate Elements" code that generates the Pages.g.cs file.

It needs to live on all machines that:
- Build a Test Studio test project
- Executes a Test Studio test

Since our installer installs this file for you, there never is a need for you, the customer, to manually install the file. Just be sure you have installed some version of Test Studio (most likely the Run-Time edition) on both your build server and your test agent machines.

Lastly no, it does not need to be added as a deployment item by your test.

I have sent you a meeting invite.

Best wishes,
Cody
the Telerik team

[Jamie]

First, one of our agents is Windows XP SP3 agenct machine, and that file is NOT on there.  I uninstalled and resintalled the Telerik.TestExecution.2011.1.502 setup, and it still was not installed.  Note, there is no Program Files (X86) folder on this computer because it is 32-bit OS.

Second, I checked our Windows 7 SP1 agent machine, and that file is NOT there either.

I believe if I understood your previous reply that file should be installed by that setup, and it appears it is not.

[Cody]

Hi Jamie,

Thank you for the additional information. The path I gave you was for VS 2010 on a 64 bit machine. For 32-bit machines use "C:\Program Files" instead of "C:\Program Files (x86)". In addition these are the folders to use for the different versions of Visual Studio:

VS2010 - "...\Microsoft Visual Studio 10.0\...."
VS2008 - "...\Microsoft Visual Studio 9.0\...."

Can you recheck in the respective folders based on your VS version and OS?

If you're using VS 2008 (though I see in your original message you stated "VS 2010), we require that Visual Studio 2008 Service Pack 1 be installed else we won't install support for Visual Studio i.e. ArtOfTest.WebAiiVSIP.dll will not get installed. VS 2010 does not have a similar requirement.

Since you seem to be using VS 2010, I'd also like to get a log file of your attempt at installing it.

1) Uninstall the run-Time edition first
2) Follow these instructions for running the install while generating a log file
3) Send me a copy of the log file (placed into a .zip file first).

Kind regards,
Cody
the Telerik team

[Jamie]

First, the controller and agents do not have full Visual Studio installed on them.  They only have Visual Studio Test Agent installed on them.  These are the machines that I installed the Test Runtime setup on.  I do not want full Visual Studio installed on them since they are suppose to simulate our clients workstation.

The machine I uninstalled and reinstalled with logging is a Windows XP SP3, 32-bit

The Install Log is attached.

Is it expected we installed full VS2010 on the test controller and agents?


Note, the file does not exist regardless the folder.  Did a global search on the test agent machine and that file was not found at all.

[Cody]

Hello Jamie,

Thanks for the new information. Now I know exactly what the problem is (I don't even have to look at the log file). Our Run-Time edition won't install support for running tests via Test Agent without it first finding full Visual Studio installed. That's why ArtOfTest.WebAiiVSIP.dll and associated registry keys aren't getting installed by our installer.

We have not yet added support for executing Test Studio tests on a pure Visual Studio Test Agent machine. It is something we can investigate supporting. My initial 5 minute investigation looks like this might be easy for us to do... but it needs to actually be tested and then have support added to our run-time installer.

Right now today you have two options:
- Install full Visual Studio on the test agent, along with re-installing our run-time
- Find a way of using our ArtOfTest.Runner.exe on the test agent machine, which does not use ArtOfTest.WebAiiVSIP.dll

Greetings,
Cody
the Telerik team

[Jamie]

I have installed Visual Studio 2010 on both the controller and agents, and I am able to Run tests from my workstation through VS code and they now execute on the agents.  I believe this is the answer, however I not believe this to be the final solution.  I would like to enter a feature request to have this issue looked at more, and offer the same functionality without having to install a full version of Visual Studio on the controller and agents.

[Cody]

Hi Jamie,

Agreed this is not the optimum solution... it is the best we can offer today. I can see the feature request you entered on this. We will log it and respond here soon.

We appreciate your patience and understanding as we constantly listen to feedback from our customers and work to improve the quality and usefulness of all our products. I have granted you additional Telerik Points in compensation for your caused inconvenience.

Greetings,
Cody
the Telerik team