Does JuckMock support mocking the static classes in unit test that runs in parallel?

[Michael]

Does JuckMock supports mocking the static classes in unit test that runs in parallel? 

I know JuckMock supports Elevated Mocking but does it work in unit tests that runs against each other in parallel?

 

[Stefan]

Hi Michael,

By default, static and future arrangements are not activated on threads other than the one doing the arrangement. To make an arrangement active on all threads, use the .OnAllThreads() clause, like so:

Mock.Arrange(() => DateTime.Now).Returns(new DateTime()).OnAllThreads();

The arrangement will remain active until the end of the current test, or until Dispose() is called on the arrangement.

Regards,
Stefan
Telerik

[Michael]

I am not looking for a way to make the static class available in all threads. When we have a static class, its available to every classes within an appdomain but it is not an issue when we run the test one by one in sequential order.  but if you are using the test framework that supports running tests in parallel, it become a problem. For example: Test1 from TestClass1 might set 1 to the static class but Test2 from TestClass2 overrides the value because it's the static class.. 

I am looking for a way to limit the scope of the static class that we mocked. I tried Microsoft Fakes. It doesn't work in parallel tests. I wonder if TypeMock supports this. 

I know using the static class or servicelocator is not good but well.. 

 

 

[Stefan]

Hello Michael,

Unfortunately, JustMock cannot be safely used when unit tests are running in parallel. The reason for that is that running tests in parallel precludes you from being able to use static and future mocking in asynchronous tests, i.e. tests that themselves run on several threads. This heavy restriction makes it impractical to invest in support for tests running in parallel.

The nearest alternative is to split your tests into multiple test containers, run several instances of the test runner in parallel and then combine the results. The upside of this approach is that there will be much less contention in JustMock so tests will run at maximum speed. The best performance will be reached if every test container can be run in the same process but in a separate app domain. It depends on the test runner whether such a mode of operation is supported.

Regards,
Stefan
Telerik

[Michael]

Hi Stefan,

Thanks for your explanation. I did a test with JustMock and seems like its working fine.  I even did a blog post for that. http://michaelsync.net/2015/10/01/unit-test-parallel-execution-of-static-classes-or-servicelocator 

I will update your comment in my post. Thanks!  

  

 

[Michael]

One restriction that I found is that I can't move this code "var data = ServiceLocator.Current.GetInstance<IDataStore>().GetData();
" to constructor (xunit uses the constructor as a setup for each test) . I tried running tests a few times and it works. but I guess it is not safe to use as you suggested it.

[Michael]

Sorry for posting different threads since I can't edit my posts. 

 Could you please look at my sample and give me your thoughts on why it's working? I did try with Microsoft Fakes and it's not working since Microsoft said that MS Fake doesn't have thread affinity. As my tests are working with JustMock, it seems like JustMock has thread affinity.. Could you please confirm that? 

[Stefan]

Hello Michael,

I took a look at your code. Yes, static and future arrangements in JustMock have thread affinity, but for a different reason - it is to prevent you from accidentally crashing the test runner. If the test runner is doing something in a separate thread, while the tests are running, arranging something that the runner depends on may very well crash it. JustMock gives you the ability to disable thread affinity on specific arrangements using the .OnAllThreads() clause.

I took a look at the code now, and it does look that JustMock can do static mocking in parallel unit tests, so I think that you should be safe. My previous statement that this is outright unsupported was not correct.

That said, if you want to write asynchronous tests, e.g. ones using async/await and Tasks, and also have static/future mocking, then you will need to use the .OnAllThreads() clause. And as soon as you do that, you can no longer run tests in parallel that use the same static members, because arrangements will bleed out into unrelated tests that happen to be running at the same time. So, it's not that parallel unit tests are unsupported, but due to the nature of mocking shared members, it's very limiting (i.e. prevents you from testing async code).

I hope the above explanation is comprehensible. If you have any further questions, I will do my best to get you an answer.

Regards,
Stefan
Telerik

[Michael]

Sure. Thanks Stefan! 

[Rob]

Hello,

does the limitation on static/future mocking for async methods and parallel test execution still apply? If so do you have an example project with this kind of setup described below:

[quote]The nearest alternative is to split your tests into multiple test containers, run several instances of the test runner in parallel and then combine the results. The upside of this approach is that there will be much less contention in JustMock so tests will run at maximum speed. The best performance will be reached if every test container can be run in the same process but in a separate app domain. It depends on the test runner whether such a mode of operation is supported.[/quote]

Regards,

Robert

[Mihail]

Hello Rob,

The limitation still applies.

Regarding the example project, we do not have such. To achieve this effect you could create one test project with all async tests and run those tests sequentially. All other tests could be in different tests projects and could be run in parallel. 

Regards,
Mihail
Progress Telerik