Issues with TestStudio 2014.1.410

[Radhika]

Hi,

We are trying to upgrade from TestStudio 2012.2.1420 to 2014.1.410. We also upgraded to Visual Studio 2012 Ultimate. We are running our tests on a remote server from Test Manager. I also installed Visual Studio 2011 Ultimate as it was complaining on missing dll as shown below.

[05/15 10:48:15,QTController.exe(11648:11),Warning] Loader.IndexLoadedSerializationTypes() : EXCEPTION! (see below)
     Situation: Exception trying to enumerate know types for assembly ArtOfTest.WebAiiVSIP, Version=2014.1.410.0, Culture=neutral, PublicKeyToken=14176981d1873d86
     Outer Exception Type: System.Reflection.ReflectionTypeLoadException
     Message: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
     HRESULT: 0x80131602 (Official ID (if app.) = COR_E_REFLECTIONTYPELOAD, Error Bit = FAILED, Facility = FACILITY_URT, Code = 5634)
     Call Stack:
          at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
          at System.Reflection.RuntimeModule.GetTypes()
          at System.Reflection.Assembly.GetTypes()
          at ArtOfTest.Common.Design.Loader.IndexLoadedSerializationTypes(Assembly assembly)     LOADER EXCEPTION #1
          Outer Exception Type: System.IO.FileNotFoundException
          Message: Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.Vsip, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
          HRESULT: 0x80070002 (Official ID (if app.) = 2147942402, Error Bit = FAILED, Facility = FACILITY_WIN32, Code = ERROR_FILE_NOT_FOUND)
     LOADER EXCEPTION #2
          Outer Exception Type: System.IO.FileNotFoundException
          Message: Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.Vsip, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
          HRESULT: 0x80070002 (Official ID (if app.) = 2147942402, Error Bit = FAILED, Facility = FACILITY_WIN32, Code = ERROR_FILE_NOT_FOUND)
     LOADER EXCEPTION #3
          Outer Exception Type: System.IO.FileNotFoundException
          Message: Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.Vsip, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
          HRESULT: 0x80070002 (Official ID (if app.) = 2147942402, Error Bit = FAILED, Facility = FACILITY_WIN32, Code = ERROR_FILE_NOT_FOUND)
     LOADER EXCEPTION #4
          Outer Exception Type: System.IO.FileNotFoundException
          Message: Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.Vsip, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
          HRESULT: 0x80070002 (Official ID (if app.) = 2147942402, Error Bit = FAILED, Facility = FACILITY_WIN32, Code = ERROR_FILE_NOT_FOUND) 

After i installed Visual Studio 2010 Ultimate the above exception was not longer thrown. No sure why we need Visual studio 2010.
 
The main issue is when we run a single test, the test starts executing immediately but once we add about 50 tests to the suite the tests wait in the queue for nearly 4 hrs before it actually starts executing the tests. We are not sure how to debug this issue. I am attaching the log from Telerik and also a screen shot from Test Manager when it is waiting in queue for 4 hrs. Please let us know if there is anything we should be looking at.

Thanks

[Cody]

Hello Radhika,

When you installed 2014.1.410, was 2012.2.1420 already installed, so it did an upgrade? Was Visual Studio 2012 Ultimate already installed? What about Visual Studio 2011 Ultimate, was it previously installed them removed?

I think at this point we need to completely uninstall Test Studio. uninstall Visual Studio 2011 Ultimate (leave only Visual Studio 2012 Ultimate left on the machine) then re-install Test Studio 2012.2.1420. The symptoms sounds like the Test Studio plug-in for VS did not get properly installed and registered.

I don't know if we can explain the 4 hour delay in the queue. That's 100% Microsoft land at that point.

Regards,
Cody
Telerik

 

[Radhika]

I uninstalled VS2010 and installed Test Studio 2012 , rebooted the server and ran tests. The tests waited for few minutes in queue which was normal.

Then  I uninstalled Test Studio 2012 and installed Test Studio 2014 and ran the upgraded solution. The tests waited in queue for 4 hrs in TM. I am not sure if this Microsoft issue. I have done this several times so far downgrading Test Studio to 2012 and everything works fine but when I upgrade the issue is back again.

Let me know if you think we need to open a support ticket as we need a fix asap.

System configuration:
Visual Studio 2012
Test Studio 2014.1.410
Visual Studio Test Agent and Controller 2010
Windows Server 2008 R2

Thanks

[Cody]

Hi Radhika,

I had my QA team test this using Visual Studio 2012 and Microsoft Test Manager. We cannot reproduce any long delays of tests waiting in the queue as you report. I wonder if it's project or test specific. Can you create a brand new project with just one very simple test, a 3 step test (Navigate and 2 clicks) and let me know how that behaves?

Regards,
Cody
Telerik

 

[Radhika]

Hi,

This happens only when you have more than 50 tests in the test suite. If you have a single test then there is no wait.
When I downgraded to Test Studio 2012, all 50 tests just waited for 10minutes in queue on Test Manager. It only happened when I upgrade to Test Studio 2014.

[Cody]

Hello Radhika,

This happens only when you have more than 50 tests in the test suite...

We discussed this in our team. Our suspicion is that one of those tests in the list is somehow causing the large delay. We can prove this theory by:

• Create a new test project
• Record a new simple 3 step test in it
• Copy and paste it within Test Studio 50 times
• Put these 50 tests into a test suite and execute

Do you get the same delay? We suspect you will not.

Regards,
Cody
Telerik