JML issue with 64-bit builds

[Isaac Abraham]

I've found an issue between Just Mock Lite when running unit tests that consistently fail, but only under Release build (presumably because of optimized code) and X64 builds with the XUnit test framework. I can't comment on whether this issue occurs under other test frameworks or not, but it definitely goes away under x86 and Debug builds. I have no idea why this happens. Cheers. Here's a sample: -

public class MyClass
{
    private readonly MockClass generator;
    public MyClass(MockClass generator)
    {
        this.generator = generator;
    }

    public void Foo(Object someData)
    {
        generator.Bar();
    }
}

public interface MockClass
{
    void Bar();
}


public class TestClass
{
    private readonly MockClass generator;
    private readonly MyClass myClass;

    public TestClass()
    {
        generator = Mock.Create<MockClass>();
        myClass = new MyClass(generator);
    }

    [Fact]
    public void X64Failure()
    {
        myClass.Foo(null);

        Mock.Assert(() => generator.Bar());
    }
}

Here's the test failure message: -

X64Failure has failed

Occurrence expectation failed. Expected calls in range [1, (any)]. Actual calls: 0

\x8Ž\x2.\x19\x3.\x7
    \x12\x1A\x3(String —)


\x2.—\x2
    Assert(Nullable`1 •\x2, Nullable`1 –\x2, Int32 \x18\x3)


Telerik.JustMock.Core.MocksRepository
    \x13˜\x2(œ\x2 œ\x2, Boolean \x11˜\x2, Occurs š\x7\x2)
    Assert(Object ›–\x2, Expression ›•\x2, Occurs š\x7\x2, Boolean \x11˜\x2)


Telerik.JustMock.Mock.\x3‚
    šž\x2()


“•\x2.’•\x2
    Š•\x2(Action ‹•\x2)

[Stefan]

Hi Isaac,

This is a known effect with the x64 runtime. The problem is that the Mock.Assert in your case is invoked using a tail call, not a regular call. This pops the test method off the call stack and Mock.Assert fails to resolve its context properly. The tail call optimization is performed only by the x64 JIT, hence the results you see. The same problem will occur with any test framework. This is only an issue with JustMock Lite, because in the full version the profiler is able to circumvent this situation.

Here's a workaround. I changed your test like so:

[Fact, MethodImpl(MethodImplOptions.NoOptimization)]
public void X64Failure()

and it started working correctly. Alternatively you could compile your tests for x86. The JIT there doesn't perform tail call optimization.

I'll add this issue into our bug database. JustMock Lite shouldn't choke like that on the very common scenario of ending your test with a Mock.Assert().

Thank you for reporting this issue. As a token of gratitude I've given you some Telerik points.

Regards,
Stefan
Telerik

[Isaac Abraham]

Hi Stefan,

Thanks for the explanation. In the short term we've had to switch our CI and published builds to Debug mode which is not ideal. Is there a plan to fix this (somehow)? I understand what you're saying re: TCO, but I've never seen this issue in any other mocking framework - so there must be a way around this.

Cheers!

Isaac

[Isaac Abraham]

If it helps at all, older versions of JML did not suffer from this issue. Only when I upgraded to the new major version of JML did we start seeing this problem.

[Kaloyan]

Hi Isaac Abraham,

Thank you for the feedback.

This is now in our backlog and it should be fixed in the very near future. Unfortunately, I am not able to give a more exact time frame.

Thank you for the understanding.

Regards,
Kaloyan
Telerik

[Isaac Abraham]

That's fine. Thanks for the update.

Rgds

Isaac

[Stefan]

Hi Isaac,

I fixed the problem with tests ending in Mock.Assert using JML in x64 Release test runs. The fix will be made available in the next internal build of JustMock.

The fix, however, doesn't pertain to Mock.AssertSet or to any non-assert method of the Mock class. If you have problems with any of those, write me back and I'll come up with a solution.

If you encounter other problems related to x64 Release test runs, kindly write us again.

Regards,
Stefan
Telerik

[Stefan]

Scratch that, I managed to fix it in all cases. The fix will be provided in the next internal build due Friday or Monday next week.

Still, if you encounter other problems related to x64 Release test runs, kindly write us again.

Regards,
Stefan
Telerik