This is a migrated thread and some comments may be shown as answers.

Mock.Create performance issue with a large interface

19 Answers 183 Views

JustMock Free Edition

This is a migrated thread and some comments may be shown as answers.

19 Answers, 1 is accepted

answered on 25 Oct 2013, 02:40 PM

Hello J,

I tried mocking an interface with 1000 methods, each with 11 parameters, and it took only 3 seconds for the mock to be created.

public interface IHugeInterface

{

void Do0(object x, int a0, int a1, int a2, int a3, int a4, int a5, int a6, int a7, int a8, int a9);

void Do1(object x, int a0, int a1, int a2, int a3, int a4, int a5, int a6, int a7, int a8, int a9);

void Do2(object x, int a0, int a1, int a2, int a3, int a4, int a5, int a6, int a7, int a8, int a9);

....

Could you share more details about that IJobInterface that takes so much time? If it's not a problem, you could just send us its source and the source of all dependencies.

Regards,
Stefan
Telerik

Share what you think about JustTrace & JustMock with us, so we can become even better! You can use the built-in feedback tool inside JustTrace, our forums, or our JustTrace or JustMock portals.

answered on 28 Oct 2013, 03:33 PM

Hi J,

I referenced your interop assembly and tried the following code:

[TestMethod]

public void TestPerformance()

{

var t = DateTime.UtcNow;

var mock = Mock.Create<IJobInterface>();

Mock.Arrange(() => mock.GetSignalCount()).Returns(123);

Assert.Equal(123, mock.GetSignalCount());

var t2 = DateTime.UtcNow - t;

}

On my machine, the test passes in about 1.5 seconds. I tried both with and without the profiler enabled. Could I ask you to profile the unit test and see what takes so much time?

Regards,
Stefan
Telerik

Share what you think about JustTrace & JustMock with us, so we can become even better! You can use the built-in feedback tool inside JustTrace, our forums, or our JustTrace or JustMock portals.

answered on 29 Oct 2013, 09:19 AM

20sRun.png

20sRun-New.png

Hi Stefan,

after some test runs I've figured out the following behaivior for your TestPerformance method:

1. Run unit test in VS08 - duration 20 seconds, almost no memory consumption
2. Debug unit test in VS08: duration 20 minutes then OutOfMemoryException

The run with the ANTS Performance profiler leads to the same result as 1. (20s, no memory consumption). But it still takes much longer than 1.5s on your machine..
Also I figured out that high memory consumption cames from the VS08 process (devenv.exe) neither VSTestHost.exe nor MSTest.exe.

I couldn't profile the high memory consumption issue if you know a way to do this please tell me. I've attached a profiler screenshot of the 20s run.

Cheers

PS: Sorry the first attachment was the wrong screenshot but it's quite the same.. 20sRun-New.png is the right one.

answered on 29 Oct 2013, 11:17 AM

Hi J,

I was able to reproduce the 20s test run time. It appears to happen only when running in the .NET 2.0/3.5 runtime. On .NET 4.0 there is no such problem.

The good news is that it happens only for the first mock created for a given type. There was almost no difference in run time between creating one mock of IJobInterface and a hundred. In other words, the 20s performance penalty is constant, regardless of the number of unit tests run.

I'm still investigating the root cause of this issue. At this point I cannot promise that the performance of Mock.Create<T> for interfaces with a large number of members running on .NET 3.5 will be improved.

If the high memory consumption is coming from devenv itself, then a profiler will probably be of little use. Your test is probably hitting some hot path in the debugger or some VS add-in. I suggest you turn to general troubleshooting techniques to resolve this issue, as it appears to be unrelated to JustMock itself. I successfully debugged the test running on .NET 3.5 with the VS2013 debugger. So it seems that it might be a bug in VS2008.

I hope that the above helps you move forward. If there's anything else that I can help you with, don't hesitate to write us back again.

Regards,
Stefan
Telerik

Share what you think about JustTrace & JustMock with us, so we can become even better! You can use the built-in feedback tool inside JustTrace, our forums, or our JustTrace or JustMock portals.

answered on 29 Oct 2013, 01:21 PM

Hello J,

Upon further investigation it appears that the VS2013 debugger also runs out of memory, so it's not a VS version issue. Also, the difference in behavior between .NET 3.5 and .NET 4.0 was caused by the way interop types get embedded. If I run the test on .NET 4.0 without embedding the interop types, then the behavior is the same - a lot of waiting ensues. The same happens with a synthetic interface.

The main problem here is that IJobInterface doesn't have 600 members - it has 6000. It implements about 15 versions of the interface, each with about 600 members and the grand total of members to mock goes to about 6000. When I tried to mock a synthetically generated interface with 6000 members, the runtime went from a second to over a minute (compared to the same interface with 1000 members).

I will continue investigating both the debugger issue and the performance issue.

Regards,
Stefan
Telerik

Share what you think about JustTrace & JustMock with us, so we can become even better! You can use the built-in feedback tool inside JustTrace, our forums, or our JustTrace or JustMock portals.

answered on 04 Nov 2013, 09:52 AM

Hello J,

In the next JustMock release (due this week) you will be able to filter which methods should be intercepted. Here's a sample how to do it:

var mock = Mock.Create<e3.IJobInterface>(conf => {

conf.SetInterceptorFilter(mi => mi.DeclaringType == typeof(e3.IJobInterface));

});

In the above code I specified that I only want to intercept methods that are declared by the IJobInterface itself, and not by any of the old version interfaces that it inherits from. The above code drastically reduces the time to create the mock type as well as the memory usage in the VS debugger when stepping over.

I sincerely hope that this helps your scenario.

Regards,
Stefan
Telerik

Share what you think about JustTrace & JustMock with us, so we can become even better! You can use the built-in feedback tool inside JustTrace, our forums, or our JustTrace or JustMock portals.

answered on 21 Nov 2013, 07:08 AM

Hello

I could test it now but still get an exception:

Test method JustMockTestProject.JustMockTestClass.GetFlagnoteIdsWithJustMock threw exception:  System.ArgumentException: Do not supply contructor arguments when mocking an interface or delegate..

at Telerik.JustMock.Core.MocksRepository.Create(Type type, MockCreationSettings settings)

at Telerik.JustMock.MockBuilder.Create(MocksRepository repository, Type type, Object[] constructorArgs, Nullable`1 behavior, Type[] additionalMockedInterfaces, Nullable`1 mockConstructorCall, IEnumerable`1 additionalProxyTypeAttributes, List`1 supplementaryBehaviors, List`1 fallbackBehaviors, List`1 mixins, Expression`1 interceptorFilter)

at Telerik.JustMock.Mock.<>c__DisplayClass69`1.<Create>b__68()

at Telerik.JustMock.Core.ProfilerInterceptor.GuardInternal[T](Func`1 guardedAction)

at Telerik.JustMock.Mock.Create[T](Object[] args)

at JustMockTestProject.JustMockTestClass.GetFlagnoteIdsWithJustMock() in JustMockTestClass.vb:line 56

I tried the line Stefan mentioned...

var mock = Mock.Create<e3.IJobInterface>(conf => {

conf.SetInterceptorFilter(mi => mi.DeclaringType == typeof(e3.IJobInterface));

});

..and converted it to VB:

mJob = Mock.Create(Of IJobInterface)(Function(conf) conf.SetInterceptorFilter(Function(mi) mi.DeclaringType Is GetType(e3.IJobInterface)))

Is that correct?

Cheers

Accepted

answered on 25 Nov 2013, 09:19 AM

Hello J,

I just tried the following test inside a .NET 3.5 VB test project, referring the e3 assembly:

<TestMethod()>

Public Sub TestMethod1()

Dim mJob = Mock.Create(Of IJobInterface)(Sub(conf) conf.SetInterceptorFilter(Function(mi) mi.DeclaringType Is GetType(e3.IJobInterface)))

End Sub

It compiled and passed inside Visual Studio 2012.

I assume the compiler of VS2008 does not allow the Sub usage inside .NET 3.5 VB lambda expressions. Can you please check if this is passing on your side in VS2010 or later?

Thanks.

Regards,
Kaloyan
Telerik

Share what you think about JustTrace & JustMock with us, so we can become even better! You can use the built-in feedback tool inside JustTrace, our forums, or our JustTrace or JustMock portals.