UnitTests very slow while JustMock Profiler enabled

[Hendrik]

Hi,

I'm using JustMock to mock SharePoint objects in some Unit Tests. When I run Unit Tests (I use ReSharper as test runner) and the JustMock profiler is enabled, the unit tests are much slower. Normally, one Unit test runs within seconds. With the JustMock profiler enabled, I have to wait several minutes. Is there something I can do to accelerate this?

 

Thank you very much!

 

[Svetlozar]

Hello,

I am really sorry for the inconvenience!

When you enable the profiler, the CLR basically calls JustMock on every method call where we check if the invoking method is arranged by JustMock. We have identified some areas where we can definitely improve in terms of performance, but on your side you can't do anything right now. I am really sorry about that. 

If you have other questions, please don't hesitate to write back!

Regards,
Svetlozar
Telerik by Progress

[Marcel]

Has there been any changes when it comes to performance? We are currently still using Telerik JustMock 2015.2.715 because we need VS2008 and .NET 3.5 support (compact framework...).

Execution times never have been a problem so far. But now we're hitting a time-critical amount of tests (1600+). My findings are that we operate at 3 tests/second and extrapolating the amount of tests currently added every month in about 1 to 2 months our test suite will hit the 30 minute mark for building + testing. Since we're letting the tests run for every feature branch these times are becoming critical for our success.

So my question is: was that addressed in a newer version of JustMock? If so: what are your findings when it comes to performance? Increasing the speed to about 10 tests/second would absolutely suffice for us. It would be a real pain in the neck to switch to a new visual studio version just to write and execute the tests but I would be willing to do it. The other option would be to stop adding more tests which is worse than anything else and would bring back long forgotten days of missed bugs and uncertainty about changes in the code base.

 

Regards
Marcel

[Mihail]

Hello Marcel,

I am afraid that the performance of JustMock is not improved significantly. The main reason for this is the call from the CLR to JustMock profiler for every executed method.

Nevertheless, we will continue looking for opportunities for improving the performance.

I am not sure if this is an option in your scenario but you could add more virtual or physical machines to your CI. This way when a build is triggered it will be executed on the next free machine and the builds won't pile up and of course, the result of each build would be known faster.

I will be happy to answer any further questions you may have.

Regards,
Mihail
Progress Telerik

[D]

I'm also interested in having better performance using the JustMock profiler. In my environment, profiling adds a 20-30s startup cost. Is this on the horizon?

[Mihail]

Hi D,

Thank you for your feedback. By receiving feedback from our clients we know what issues they have and where we should focus. So keep it coming.

We have some ideas on how we could further optimize how the methods are instrumented by the JustMock profiler. I have opened a new feature request in our feedback portal where you could upvote it and subscribe to status changes. Once we start working on this optimization we will update the item.

Regards,
Mihail
Progress Telerik