Record -> Play

[Tormod]

Hi.

My projects depend on a big common.dll that have many dependencies. Amongst others, it references System.Environment, but I'm not sure when and what. Currently I have set up the windows environment variables so that the tests execute, but I don't want to depend on that when running my unit tests for the future.

Can I mock out System.Environment, run a test case and then inspect intercepted calls to it so that I can create a sensible mocking behaviour?

----

Also, sometimes, especially from legacy components, I get a rather sophisticated blob. For example from an automated fire central. In order to testrun my application, I normally need to be connected to the real fire central.

I would like to turn these integration tests scenarios into unit tests. And I'm tired of creating test scenarios manually on the fire central.

The approach that makes the most sense to me, is that I would be connected the real HW and mock the fire central communication class, then I would run a complete scenario and then inspect the mock for what calls were being made to it and what the response from the fire central was, including events. Then I wish to easily arrange a mock that reproduces this behaviour for the future without the fire central connected.

Is there anything in JustMock that facilitates this?

[Ricky]

Hello Tormod,

Thank you for your query. Regarding the first question of whether you can mock System.Enviroment or more specifically msCorlib members like File or DateTime. Yes it is possible. Just to show how it can be done provided that I am using NUnit (Same can be achieved with other test tools as well). Let’s consider the following example:

We want to mock Enviorment.Exit(int) call . Therefore during the test initialization, we first need to setup the mock using Mock.Partial since this is a mscorilb member and should be considered specially. For some commonly used members such as File, DateTime , you don’t require this step but it is required to mock members from System.Enviroment.

[TestFixtureSetUp]
public void Initialize()
{
    Mock.Partial(typeof(Environment)).For<int>((i) => Environment.Exit(i));
}

Just to note that there were a few signatures missing in the Mock.Partial call but they will be bundled though with the service pack release in a couple of days with other enhancements.

Once, it is done. The rest is as usual that follows:

[TestMethod]
public void ShouldMockMembersFromEnviroment()
{
    bool called = false;
    Mock.Arrange(() => Environment.Exit(10)).DoInstead(() => called = true);
    Environment.Exit(10);
    Assert.True(called);
}

To  find out more examples on mocking mscorlib members, please navigate to  \%PROGRAMFILES%\Telerik\JustMock\ Examples\ and check the MsCorlibFixture.cs under Elevated folder of either NUnit or MSTest examples.

Regarding your second question that whether you can fake out the responses from a centrally located repository which are pretty straight forward and can be done using JustMock.

Regards,
the Telerik team

[Tormod]

Hi and thank you for the answer.

I'm afraid that it wasn't quite what I was after, though.

I know that I can mock predicted calls to System.Environment methods. 
But what I want to do is to place a strict mock on System.Environment as a whole.

I am calling a library which depends on System.Environment, but I'm not quite sure how or which methods are called during my test execution.
And I do not know wether it depends on System.DateTime.

I wish to proceed as follows

• Place the strict mock on System.Environment.
• Run my test code.
• Get an exception because an unannounced call to System.Environment.GetEnvironmentVariable was made.
• Add an expectation and a proper mock return value on that specific invocation.
• Retry.
• Get another exception because a call to System.Environment.GetSystemFolder was made by the same library.
• Repeat until I have successfully mocked out all calls to System.Environment for the test scenario.

The second case is kind of the same, but the point here is observe the behaviour of the real system and create sophisticated return data.

• I start out with a test project with external dependencies and externally connected test hardware. Let's say that it is a fire central connected over a serial line.
• I do not want my tests to depend on being connected to a fire central over the serial line, but initially this is the only thing in the world that is capable of producing the correct callbacks from the driver library that came with it.
• I wish to create a mock of the fire central proxy object. The mock is set up to record the behaviour of the mocked object while still calling the original object.
• Then I run a test scenario which includes me pushing test buttons on the physical fire central. This is something I do not wish to do every time I need to test my code.
• Then I wish to have an easy way of using the recorded behavior to create a mock that reproduces the behavior so that the unit test no longer depends on the real object to run the same scenario.

This is a common case when you communicate with drivers that, for example, produce byte arrays with data. It would also allow me to create integration tests that verify that the behavior of the HW hasn't changed whenever a new version of it is released.

[Ricky]

Hello Tormod,

Thanks for your reply. Now, about your first question of making a strict mock of Environment class. As this is a framework class (belonging to the MsCorlib assembly), you need to do Mock.Partial (as mentioned in the last post) during the test initialization. Unfortunately, it's not currently possible to create strict mock of the Environment class. But we have a plan to add the following syntax for Mock.Partial so that it is also possible to create strict mock of Environment like classes using Mock.SetupStatic(typeof(Environment), Behavior.Strict).

Mock.Partial(typeof(Environment)).ForAll();

After this, in the test class you will be able to create a strict mock of the Environment class through Mock.SetupStatic as specified above. This is queued in the backlog already and hopefully it will be available soon.

Secondly, it seems from your specification that you are looking for a Record -Replay support in Justmock. JustMock is based on AAA syntax that is simple and easy to read and maintain, whereas Record-Replay is quite opposite to what AAA stands for. You might like to check the following interesting posts on this:

http://stackoverflow.com/questions/650494/mocking-using-traditional-record-replay-vs-moq-model
http://www.clariusconsulting.net/blogs/kzu/archive/2007/12/26/48177.aspx

But, there is a good news that we have decided to add the Record-Replay support to JustMock as well under Telerik.JustMock.Legacy namespace which will also be available soon.

Hope this information helps.

Kind regards,
the Telerik team

[Tormod]

Hi.

 

Regarding item 1: That is great news. Thank you. It will save me an awful lot of guesswork in what framework classes are being accessed by external libraries. I will just run and let it crash.

 

Regarding item 2: I apologize for the confusion. I am not looking for Record->Replay as is implemented in other mocking framework. I am completely onboard AAA wagon.

The standad record->replay model is to create a mock, set it in record mode and then act upon it according to your expectations of how the object will be acted upon by the surroundings.

 

I am talking about the complete opposite. I wish to wrap the real object and then I want to not only record the interaction upon it, but also how the real object responds. Then I wish to be able to (manually) create a standard Arrange-Act-Assert unit test whose "Arrange" section composes a mock that reproduces the behavior that was recorded.

 

For example:  I have a third party library that, when connected to the live system, passes me byte arrays and strangely formatted strings. It is tedious to create unit tests with mocks that reproduces this behavior.

 

I can have access to the live system for a few days and I wish to spend that time recording scenarios. Then I wish to go back to TDD and interact upon JustMock mock objects that pass me the same datagrams without the unit tests depending on the live system or the third party library.

 

It's not a feature I've seen with any other mock library so I completely understand if the feature seems misplaced, but I imagine that it is a common problem when interacting with legacy libraries.

[Ricky]

Hello Tormod,

Thanks again for your reply.
According to what you are saying here is that you want to simulate a recoding session in live environment that will take account of all the faults and possible scenarios of how and where things are failing. Then analyze it later in your dev environment to figure out what could have gone wrong and write possible test methods for it.

If that's correct I guess for this case it will be better to use a logging tool to log all the failing cases then analyze it using some log parser and finally on top of that write all the tests that you think can replicate the defects to fix.

Of course all the workflow associated with doing this could be automated somehow and we could definitely consider this as part of JustMock or maybe as a separate module.

Thanks a lot for suggesting that!

Hope that helps,
the Telerik team

[Tormod]

Yes, that is basically it. But it's not specific to failure scenarios. It's not a debugging tool, although it could be useful as that as well. This is how I would go about creating my "Arrange" sections in my normal TDD activity.

I have a an object X, which has a behavior. This behavior may depend on X being part of a live system. Any specific scenario may depend on me monkeyeing around pressing buttons or otherwise stimulating input on the external system. Today, we have manual test scripts that are inches thick and the system setup takes days.

I wish to create a Mock of X that will able to reproduce behavior of X by monitoring it.
It then makes sense that I would create a mock of X and pass it a reference to a properly configured and live connected object X. After doing just one thing one the live system (either press a button, pull out a cable, stimulate a sensor, add a sensor, just letting it run in idle for a minute) I wish to break execution and then see what object X has done in the meantime. A standard profiler does this.

However, the crux of the functionality comes if, after a run of a scenario, there was an easy way to arrange a mock of X that was not connected to the live system, but which still did what the object X did from the client perspective. Instead of a live system with a manual test procedure that includes N ways of manipulating the live system, I could have N different ways of arranging a mock of X.

From a mocking vendor perspective, I don't think that this should be too hard. You mock an instance of X and then intercept method calls, property access and events, and always call the original while cloning the objects being passed back and forth. Then, I should be able to break the execution and within the visual studio IDE inspect the X wrapper to compose "Arrange" sections to have a mock X reproducing parts of that behavior.

This would have a bunch of uses.
I could also use this to verify that a protocol has not changed from one version to another.
Many scenarios are hard to reproduce on physical hardware. If I had a mock, I could tweak the timing to be just right, switch the order of two events etc.


But, like I said, I do not know of any other mocking framework that does this.
Thank you for your patience.

Regards,

Tormod

[Ricky]

Hello Tormod,

Thanks again for the explanation and sorry for the late reply. As to achieve your goal of mocking X objects as stated in your post, I would like to ask you about the issues you are facing while mocking live objects.

Furthermore, If you could share some actual code and point out some issues it would greatly help us identify the exact issue.
If your script is dealing with plan old C# objects (POCOs) and some business rule associated with them, then I think you just need to mock the objects that are making requests to the live system to run your tests in a full isolated environment.

Once again, thanks for the feedback and I'm looking forward to your reply.

Best regards,
the Telerik team