Azure Pipeline JustMockVSTest-2 Elevated not working (System.InvalidProgramException : Common Language Runtime detected an invalid program.)

[NovoPath]

Hi,

I'm trying to create a pipeline on Azure for test/coverage using JustMock, my application is Core 5.0*

After a long journey to make the build work, now I'm encountering an issue with the test (JustMockVSTestV2)

steps:
- task: vs-publisher-443.jm-vstest-2.JustMockVSTest-2.JustMockVSTest@2
  displayName: 'VsTest - testAssemblies'
  inputs:
    testAssemblyVer2: |
     **\bin\**\*test*.dll
     !**\*TestAdapter.dll
     !**\*TestPlatform*
     !**\obj\**
     !**\bin\**\ref\**

    pathTo64BitJustMockProfiler: '[correctpath]\bin\Release\net5.0\runtimes\win-x64\native\Telerik.CodeWeaver.Profiler.dll'
    pathTo32BitJustMockProfiler: '[correctpath]\bin\Release\net5.0\runtimes\win-x86\native\Telerik.CodeWeaver.Profiler.dll'
    vsTestVersion: toolsInstaller
    runTestsInIsolation: true
    codeCoverageEnabled: true
    otherConsoleOptions: '/Framework:.NETCoreApp,Version=v5.0.401 /Enablecodecoverage /logger:trx'
    platform: '$(BuildPlatform)'
    configuration: '$(BuildConfiguration)'

 

My Test functions requires elevation:

Mock.SetupStatic(typeof(AStaticClass));

var response = Mock.Create<IRestResponse<string>>(); //this is restsharp
            Mock.Arrange(() => response.Data)
                .Returns("SomeData");

var client = Mock.Create<RestClient>(Constructor.Mocked);
            Mock.Arrange(() => client.ExecuteAsync<string>(Arg.IsAny<IRestRequest>(), Arg.IsAny<CancellationToken>()))
                .IgnoreInstance()
                .TaskResult(response);

 

My test project works fine on my local machine when I enable JustMock profile, however I get a bunch of errors on Azure pipeline:

##[error][xUnit.net 00:00:03.76]     ProjectTests.GetAsync_StateUnderTest_ExpectedBehavior [FAIL]
[xUnit.net 00:00:03.76]       System.InvalidProgramException : Common Language Runtime detected an invalid program.
[xUnit.net 00:00:03.76]       Stack Trace:
[xUnit.net 00:00:03.79]            at ProjectTests.GetAsync_StateUnderTest_ExpectedBehavior()
[xUnit.net 00:00:03.79]            at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
[xUnit.net 00:00:03.79]            at ProjectTests.GetAsync_StateUnderTest_ExpectedBehavior()

ALSO

##[error]Testhost process exited with error: Cannot use file stream for [PATH\bin\Release\net5.0\testhost.deps.json]: No such file or directory
##[error]A fatal error was encountered. The library 'hostpolicy.dll' required to execute the application was not found in 'C:\hostedtoolcache\windows\dotnet'.
##[error]Failed to run as a self-contained app.
##[error]  - The application was run as a self-contained app because 'PATH\bin\Release\net5.0\testhost.runtimeconfig.json' was not found.
##[error]  - If this should be a framework-dependent app, add the 'PATH\bin\Release\net5.0\testhost.runtimeconfig.json' file and specify the appropriate framework.
##[error]. Please check the diagnostic logs for more information.

 

Please advice, I'm not able to make JustMock to work with Azure Pipeline.

 

Thanks

[Ivo]

Hello,

The following JustMock VsTest task settings required attention:

• runTestsInIsolation: true

I am not quite sure that it is directly related to the problem, but usually when enabled brings more issues than benefits, my recommendation is to keep it disabled

• codeCoverageEnabled: true

It is fine unless not duplicated in the command line below, if you need to have coverage enabled prefer to use this option

• otherConsoleOptions: '/Framework:.NETCoreApp,Version=v5.0.401 /Enablecodecoverage /logger:trx'

In my opinion, the last two options are redundant

One more thing, try to specify the filter (input setting of the task, the one with wildcards) that pickups only the test assemblies needed for execution, sometimes it might be the key in resolving the strange issues like this, please refer to this article specific to xUnit. 

I hope the provided information helps. If you are still experiencing difficulties I would be happy to assist you further.

Regards,
Ivo
Progress Telerik

[NovoPath]

Hi Ivo, 

I managed to fix the .json issue with the File Test, however the main issue is still happening:

YAML:

- task: DotNetCoreCLI@2
  displayName: Build
  inputs:
    projects: '$(Parameters.RestoreBuildProjects)'
    arguments: ' --configuration $(BuildConfiguration) --no-restore --force

- task: vs-publisher-443.jm-vstest-2.JustMockVSTest-2.JustMockVSTest@2
  displayName: 'VsTest - testAssemblies'
  inputs:
    testAssemblyVer2: |
     **\bin\**\*test*.dll
     !**\*TestAdapter.dll
     !**\*TestPlatform*
     !**\obj\**
     !**\bin\**\ref\**
     !**\bin\**\*.json
     !**\bin\**\testhost.dll
    pathTo64BitJustMockProfiler: 'PATH\bin\Release\net5.0\runtimes\win-x64\native\Telerik.CodeWeaver.Profiler.dll'
    pathTo32BitJustMockProfiler: 'PATH\bin\Release\net5.0\runtimes\win-x86\native\Telerik.CodeWeaver.Profiler.dll'
    vsTestVersion: toolsInstaller
    runInParallel: false
    runTestsInIsolation: false
    codeCoverageEnabled: true
    otherConsoleOptions: '/Framework:.NETCoreApp,Version=v5.0.401'
    platform: '$(BuildPlatform)'
    configuration: '$(BuildConfiguration)'

 

Generated .runsettings from the pipeline:

 <RunSettings>
   <DataCollectionRunSettings>
     <DataCollectors>
       <DataCollector friendlyName="Code Coverage Wrapper" uri="datacollector://Microsoft/CodeCoverage/2.0">
         <Configuration>
           <SinkDataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" isLinked="true">
             <Configuration>
               <CodeCoverage>
                 <ModulePaths>
                   <Include>
                     <ModulePath>.*.dll$</ModulePath>
                     <ModulePath>.*.exe$</ModulePath>
                   </Include>
                   <Exclude>
                     <ModulePath>.*CPPUnitTestFramework.*</ModulePath>
                   </Exclude>
                 </ModulePaths>
                 <Functions>
                   <Exclude>
                     <Function>^Fabrikam.UnitTest..*</Function>
                     <Function>^std::.*</Function>
                     <Function>^ATL::.*</Function>
                     <Function>.*::__GetTestMethodInfo.*</Function>
                     <Function>^Microsoft::VisualStudio::CppCodeCoverageFramework::.*</Function>
                     <Function>^Microsoft::VisualStudio::CppUnitTestFramework::.*</Function>
                   </Exclude>
                 </Functions>
                 <Attributes>
                   <Exclude>
                     <Attribute>^System.Diagnostics.DebuggerHiddenAttribute$</Attribute>
                     <Attribute>^System.Diagnostics.DebuggerNonUserCodeAttribute$</Attribute>
                     <Attribute>^System.Runtime.CompilerServices.CompilerGeneratedAttribute$</Attribute>
                     <Attribute>^System.CodeDom.Compiler.GeneratedCodeAttribute$</Attribute>
                     <Attribute>^System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute$</Attribute>
                   </Exclude>
                 </Attributes>
                 <Sources>
                   <Exclude>
                     <Source>.*\atlmfc\.*</Source>
                     <Source>.*\vctools\.*</Source>
                     <Source>.*\public\sdk\.*</Source>
                     <Source>.*\microsoft sdks\.*</Source>
                     <Source>.*\vc\include\.*</Source>
                   </Exclude>
                 </Sources>
                 <CompanyNames>
                   <Exclude>
                     <CompanyName>.*microsoft.*</CompanyName>
                   </Exclude>
                 </CompanyNames>
                 <PublicKeyTokens>
                   <Exclude>
                     <PublicKeyToken>^B77A5C561934E089$</PublicKeyToken>
                     <PublicKeyToken>^B03F5F7F11D50A3A$</PublicKeyToken>
                     <PublicKeyToken>^31BF3856AD364E35$</PublicKeyToken>
                     <PublicKeyToken>^89845DCD8080CC91$</PublicKeyToken>
                     <PublicKeyToken>^71E9BCE111E9429C$</PublicKeyToken>
                     <PublicKeyToken>^8F50407C4E9E73B6$</PublicKeyToken>
                     <PublicKeyToken>^E361AF139669C375$</PublicKeyToken>
                   </Exclude>
                 </PublicKeyTokens>
                 <UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>
                 <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
                 <CollectFromChildProcesses>True</CollectFromChildProcesses>
                 <CollectAspDotNet>False</CollectAspDotNet>
               </CodeCoverage>
             </Configuration>
           </SinkDataCollector>
         </Configuration>
       </DataCollector>
     </DataCollectors>
   </DataCollectionRunSettings>
   <RunConfiguration>
     <TestAdaptersPaths>D:\a\_tasks\JustMockVSTest_dfc18792-8901-4297-a4de-6edef6796070\2.6.1\DataCollectors\JustMock\2021.1.222.1</TestAdaptersPaths>
     <BatchSize>1000</BatchSize>
     <ResultsDirectory>D:\a\_temp\TestResults</ResultsDirectory>
   </RunConfiguration>
 </RunSettings>

Generated vstest.console cli:

vstest.console.exe "D:\a\1\s\Tests\Project1.Tests\bin\Release\net5.0\Project1.Tests.dll"
"D:\a\1\s\Tests\Project2.Tests\bin\Release\net5.0\Project2.Tests.dll"
"D:\a\1\s\Tests\Project3.Tests\bin\Release\net5.0\Project3.Tests.dll"
/Settings:"D:\a\_temp\ta04r5n0xle.tmp.runsettings"
/Logger:"trx"
/TestAdapterPath:"D:\a\1\s"
/Framework:.NETCoreApp,Version=v5.0.401

All the test passes except the one that requires elevation shows:

System.InvalidProgramException : Common Language Runtime detected an invalid program

 

Many thanks

[NovoPath]

One note, When I run "Analyze Code Coverage for all Tests" from visual studio I also get:

System.InvalidProgramException : Common Language Runtime detected an invalid program.

 

[Mihail]

Hello,

Is it possible to isolate the unit test and the code that is tested in a simple test project which we can use to reproduce the issue? This way we will be able to investigate the issue in more detail.

[NovoPath]

Please find attached a sample project (ReproduceJustMock.zip)

1- When you run the tests with Profile Enabled all the test passed.

2- When you run "Analyze Code Coverage for all tests" all the tests failed.

3- Same issue on Azure Pipeline.

 

NB: Also if you use Moq in the same project, some moq tests will failed if telerik profile is enabled.

[Mihail]

Hello,

Thank you for the repo project. It was very useful.

I can confirm that the issue exists and it occurs when the code coverage is triggered inside Visual Studio. I've logged a bug report into our feedback portal. Here is a link to the item if you would like to subscribe for status changes.
The item is still with status unplanned as it is still not scheduled. Having that said, I will recommend to the team to include in for the upcoming R3 SP1 2021. Once we have more information we will update the item.

In appreciation for your involvement, I've updated your Telerik points.