Why Isolating Your Unit Tests Matters

Developers are already isolating their code, even when they aren’t running unit tests, because it lets them do more in less time. Embracing isolation when testing has those benefits and more: better code, faster development and more reliable releases.

Developers are already isolating their code, even if it isn’t something they worry about when running unit tests. When a developer debugs an application, for example, they’ll probably be running their code against a local copy of the application’s database. Not only does that cause the application to run faster when debugging (or testing), it also ensures that no one messes up the test data that they’ve built up over time.

In fact, most developers are also debugging their applications on a local web server and, ideally, have installed any associated web services on that local server. If that’s not the case, they’re calling a “development” version of those web services (if, for no other reason, because no one wants test code to be updating to the “production” version of the services).

But if everyone had what they wanted, developers would be using local copies of everything their code interacts with: they’d run against local services, on a local web server, and use a local database for their code and the services they call. Local, isolated resources give developers both speed and independence.

Of course, that requires some way to maintain that environment. The problem with running development code, for example, is that it changes your data and, usually, not in a good way. It’s always embarrassing, for example, when some delete operation goes rogue and blows away all your test data.

To handle that, developers have multiple tools for re-initializing the development environment. For test data, that may be backups of the database that can be used to restore to a “Stage Zero.” Alternatively, development code may leverage Entity Framework’s initialization options to recreate that “Stage Zero” when the code starts up. Other resources might include PowerShell scripts that recreate the test environment or Azure templates that recreate cloud resources.

Essentially, these tools isolate code in time—it doesn’t matter what happened before the code started running because a debugging session always begins from a well-defined “Stage Zero.” Ideally, there’s a single action that does everything necessary to isolate development code from whatever happened before the code starts running (and, ideally, an action that happens automatically when the code starts because, let’s face it, everyone occasionally forgets to perform a restore to “Stage Zero”).

Testing Isn’t Debugging

Creating a local development environment makes as much sense for unit tests as it does for code. But the difference between testing and debug is that, when developing code, you don’t always need a “perfect” answer so resetting to “Stage Zero” isn’t always required.

Often, when debugging, you’re happy if your code just runs to completion—you’re working on some specific area of the code rather investigating a complete transaction. Sometimes, you may not even intend to let your code run to completion: You’re just interested in finding where a problem is so that you can stop your code and resolve the problem.

Testing is different. Every unit test should be completely isolated from any other unit test because, on every test, you do need a perfect answer: Does the code work or not? You can tell if your tests are isolated by asking just one question: Does the order in which you run your tests alter the tests’ outcome? If the answer is “No” then your tests are truly isolated from each other.

As with isolating your code, there are real benefits to isolating your tests. Among other things, isolating your tests dramatically simplifies your life. If the order you run your tests does matter well, then, you have to manage the order of your tests. If the order doesn’t matter, then you can run any test (or sets of tests) without having to think about it.

If you go one step further and can also recreate the environment that your tests run in—database and web services and anything else that matters—then you get a further benefit: It doesn’t matter where you run your tests, either. You can recreate your test environment on any computer whenever you want to run your test. Effectively, then, you’ve isolated your tests in space, also. To achieve that goal, in addition to your PowerShell scripts, database backups and so on, you might also include deployment packages for the web services that the CUT (code under test) uses.

The goal of isolating your tests in both time and space is to create stable tests: Tests that give the same results no matter when they’re run, where they’re run, what order they’re run in and what else is going on at the same time. Creating stable tests isn’t trivial (if you’re not already thinking about containers, this might be a good time to start). But it’s also not hard to do and there are real benefits to gain from creating stable tests.

The Benefits of Isolation

The first benefit is help in identifying bugs. If your tests are isolated across both time and space, then, if a test fails, it has to be because of the code that the test exercises … and it will be because of something done since the last time the tests were run (there’s an incentive to run your tests frequently).

In other words, the code that’s “top of mind” right now is the code that’s causing the tests to fail. Coupled with good practices for naming tests, often just reading the name of a failing test will trigger an understanding of what’s causing the test to fail. Finding bugs faster means delivering code faster.

If multiple tests fail, then it has to be because of something those tests share. And that’s a code smell: If you can’t isolate your tests then it’s a sign that the code is probably too tightly coupled. Writing isolated unit tests forces developers to think about creating loosely coupled code—code that’s more flexible, reusable and resilient.

And, by isolating your tests across the platforms you run your tests on, you’ve also documented what your production environment has to look like—assuming, that is, you want your code to run as well in production as it does in test (and you do). The tools that isolate your tests enable you to create, in production, the same environment that allowed your code to run successfully in test.

What isolation really means is that you may never hear the dreaded words, “Well, it worked in test,” again.

Unit Test Isolation With Mocking Framework

Achieving perfect isolation isn’t trivial—that is, if you try to do it all on your own. Most developers use mocking frameworks, like Telerik JustMock, to create mocks for calls to database servers, web services, third-party libraries or even another chunk of their own code. Typically, all it requires is one or two lines of code added to the “arrange” portion of your test to isolate the component you want to test.