Isaac Abraham
Top achievements
Rank 1
Isaac Abraham
asked on 05 Aug 2013, 10:27 AM
I've found an issue between Just Mock Lite when running unit tests that consistently fail, but only under Release build (presumably because of optimized code) and X64 builds with the XUnit test framework. I can't comment on whether this issue occurs under other test frameworks or not, but it definitely goes away under x86 and Debug builds. I have no idea why this happens. Cheers. Here's a sample: -
Here's the test failure message: -
public
class
MyClass
{
private
readonly
MockClass generator;
public
MyClass(MockClass generator)
{
this
.generator = generator;
}
public
void
Foo(Object someData)
{
generator.Bar();
}
}
public
interface
MockClass
{
void
Bar();
}
public
class
TestClass
{
private
readonly
MockClass generator;
private
readonly
MyClass myClass;
public
TestClass()
{
generator = Mock.Create<MockClass>();
myClass =
new
MyClass(generator);
}
[Fact]
public
void
X64Failure()
{
myClass.Foo(
null
);
Mock.Assert(() => generator.Bar());
}
}
Here's the test failure message: -
X64Failure has failed
Occurrence expectation failed. Expected calls
in
range [1, (any)]. Actual calls: 0
\x8\x2.\x19\x3.\x7
\x12\x1A\x3(String )
\x2.\x2
Assert(Nullable`1 \x2, Nullable`1 \x2, Int32 \x18\x3)
Telerik.JustMock.Core.MocksRepository
\x13\x2(\x2 \x2, Boolean \x11\x2, Occurs \x7\x2)
Assert(Object \x2, Expression \x2, Occurs \x7\x2, Boolean \x11\x2)
Telerik.JustMock.Mock.\x3
\x2()
\x2.\x2
\x2(Action \x2)
7 Answers, 1 is accepted
0
Hi Isaac,
This is a known effect with the x64 runtime. The problem is that the Mock.Assert in your case is invoked using a tail call, not a regular call. This pops the test method off the call stack and Mock.Assert fails to resolve its context properly. The tail call optimization is performed only by the x64 JIT, hence the results you see. The same problem will occur with any test framework. This is only an issue with JustMock Lite, because in the full version the profiler is able to circumvent this situation.
Here's a workaround. I changed your test like so:
and it started working correctly. Alternatively you could compile your tests for x86. The JIT there doesn't perform tail call optimization.
I'll add this issue into our bug database. JustMock Lite shouldn't choke like that on the very common scenario of ending your test with a Mock.Assert().
Thank you for reporting this issue. As a token of gratitude I've given you some Telerik points.
Regards,
Stefan
Telerik
This is a known effect with the x64 runtime. The problem is that the Mock.Assert in your case is invoked using a tail call, not a regular call. This pops the test method off the call stack and Mock.Assert fails to resolve its context properly. The tail call optimization is performed only by the x64 JIT, hence the results you see. The same problem will occur with any test framework. This is only an issue with JustMock Lite, because in the full version the profiler is able to circumvent this situation.
Here's a workaround. I changed your test like so:
[Fact, MethodImpl(MethodImplOptions.NoOptimization)]
public
void
X64Failure()
I'll add this issue into our bug database. JustMock Lite shouldn't choke like that on the very common scenario of ending your test with a Mock.Assert().
Thank you for reporting this issue. As a token of gratitude I've given you some Telerik points.
Regards,
Stefan
Telerik
0
Isaac Abraham
Top achievements
Rank 1
answered on 14 Aug 2013, 09:36 AM
Hi Stefan,
Thanks for the explanation. In the short term we've had to switch our CI and published builds to Debug mode which is not ideal. Is there a plan to fix this (somehow)? I understand what you're saying re: TCO, but I've never seen this issue in any other mocking framework - so there must be a way around this.
Cheers!
Isaac
Thanks for the explanation. In the short term we've had to switch our CI and published builds to Debug mode which is not ideal. Is there a plan to fix this (somehow)? I understand what you're saying re: TCO, but I've never seen this issue in any other mocking framework - so there must be a way around this.
Cheers!
Isaac
0
Isaac Abraham
Top achievements
Rank 1
answered on 16 Aug 2013, 08:41 AM
If it helps at all, older versions of JML did not suffer from this issue. Only when I upgraded to the new major version of JML did we start seeing this problem.
0
Hi Isaac Abraham,
Thank you for the feedback.
This is now in our backlog and it should be fixed in the very near future. Unfortunately, I am not able to give a more exact time frame.
Thank you for the understanding.
Regards,
Kaloyan
Telerik
Thank you for the feedback.
This is now in our backlog and it should be fixed in the very near future. Unfortunately, I am not able to give a more exact time frame.
Thank you for the understanding.
Regards,
Kaloyan
Telerik
0
Isaac Abraham
Top achievements
Rank 1
answered on 16 Aug 2013, 11:47 AM
That's fine. Thanks for the update.
Rgds
Isaac
0
Hi Isaac,
I fixed the problem with tests ending in Mock.Assert using JML in x64 Release test runs. The fix will be made available in the next internal build of JustMock.
The fix, however, doesn't pertain to Mock.AssertSet or to any non-assert method of the Mock class. If you have problems with any of those, write me back and I'll come up with a solution.
If you encounter other problems related to x64 Release test runs, kindly write us again.
Regards,
Stefan
Telerik
I fixed the problem with tests ending in Mock.Assert using JML in x64 Release test runs. The fix will be made available in the next internal build of JustMock.
The fix, however, doesn't pertain to Mock.AssertSet or to any non-assert method of the Mock class. If you have problems with any of those, write me back and I'll come up with a solution.
If you encounter other problems related to x64 Release test runs, kindly write us again.
Regards,
Stefan
Telerik
0
Scratch that, I managed to fix it in all cases. The fix will be provided in the next internal build due Friday or Monday next week.
Still, if you encounter other problems related to x64 Release test runs, kindly write us again.
Regards,
Stefan
Telerik
Still, if you encounter other problems related to x64 Release test runs, kindly write us again.
Regards,
Stefan
Telerik