In this day and age, unit testing isn’t as controversial as it once was. Sure, you still see the occasional inflammatory, clickbait-y, confrontational “unit testing is garbage” type of post on Reddit and Hacker News.
And there are still developers out there ignorant of the existence of unit testing—or any type of automated testing, for what it’s worth. Believe it or not.
However, if we consider the developers that do know about unit testing, I’d say the debate is pretty much over. It’s generally agreed, today, that unit testing has a positive influence on software projects.
With that in mind, the next question then becomes “what makes for a good unit test?” That’s what this post is all about. Today, we present you five must-haves for a great unit test.
1. Speed
The first must-have for your unit testing suite is speed. I’m far from being the first one to say that, even though I’d be pleased to be the last. Household names in the industry have told us the same thing time and time again.
A slow test suite will discourage developers from running it. By not running the tests as often as possible, the developers will miss out on what’s arguably the primary benefit of unit testing: the confidence to change the code without fear.
Why are the tests slow? It could be that the tests interact with real external dependencies (e.g., the database, the filesystem, etc.) instead of faking them. Or it even may be that some algorithm was implemented inefficiently.
But it also could be that the architecture of the application isn’t clean enough. Maybe there’s too much coupling and too many dependencies, making it impossible for the test code to exercise a unit in complete isolation.
If this is the case, the slowness of the tests is doubly evil: it’s both a symptom of already-existing problems and the cause for new ones to come.
2. Focus
What’s the adequate number of assertions per test method? As many as you wish? Just one? Blind adoption of an extreme viewpoint is always…well, extreme, but it also amounts to cargo cult programming, and we know better than that by now.
The only reasonable answer to the “how many assertions per test method” question then becomes apparent: it depends. The correct number of assertions might be one for this test but three for that other and then two for another. You must use as many assertions as you need to correctly exercise that specific scenario for the unit of work under test.
Let’s see a quick example. Suppose the following snippet is a test method for a custom stack class:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
[Test] public void Push_ToEmptyStack_ShouldContainTheItem() { var sut = new CustomStack<int>(); sut.Push(21); Assert.Multiple(() => { Assert.AreEqual(21, sut.Peek()); Assert.AreEqual(1, sut.Count); }); } |
As you can see, the test has two assertions. Is it doing too much? Has it lost its focus? No, it isn’t, and no, it hasn’t. There are two assertions, but they’re both employed in documenting the same scenario. In other words, both assertions help tell the same story:
- There was an empty stack.
- We pushed an element to it.
- Now it has the element on it (note: not just “an” element but “the” element we’ve pushed).
What do we take from this? The test starts losing its focus when it tries to tell more than one story in a single test case.
3. Reliability
Your unit tests must be reliable. You must be able to trust them. Otherwise, you and your team won’t have the confidence to refactor fearlessly. Then it’s virtually guaranteed the unit tests will become less of a good thing to cherish and more of a burden to loathe.
And how do you trust your tests? Easy. You make them trustworthy. You make them reliable.
Your tests shouldn’t just start failing out of the blue. If the tests were passing yesterday and no changes were made to the code, then they should still pass today. If the tests you wrote for your open-source project work on your computer in the US, then they should also pass on the computer of a Japanese contributor. The same goes for a Turkish, Brazilian, or French contributor, of course.
When a unit test starts randomly failing, that’s a sign that it depends on things it shouldn’t depend on. It’s making assumptions about the environment it’s running on, its timezone, culture, or other constraints.
Let’s drive the point home, using a quick example. Consider the following snippet:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
public class Distance { private readonly double meters; private Distance(double meters) { this.meters = meters; } public static Distance FromMeters(double meters) => new Distance(meters); public override string ToString() => meters.ToString("F") + " m"; } |
It’s a quintessential value object: a distance class. It’s a very rudimentary implementation, though. A real implementation would have factory methods for other units, methods for performing arithmetic, and so on. But for our example, it will suffice.
Consider now the following test:
|
1 2 3 4 5 6 7 |
[Test] public static void ToString_ReturnsFormattedValue() { var sut = Distance.FromMeters(12.5); var expectedResult = "12.50 m"; Assert.AreEqual(expectedResult, sut.ToString()); } |
“What’s the problem with the test?” you might wonder. Don’t keep wondering. Fire up Visual Studio right now, create a new solution, copy and paste the code, run the test, and find out.
If I had to guess, I’d say the test passed…if you’re in the United States. Or Mexico, Israel, Japan, and many more countries. If, on the other hand, you’re in Brazil, France, Germany, or Argentina, among several others, the test above will fail!
And why is that? Simple. The countries in the latter list use a comma as decimal separator. Usually, the overload of ToString that doesn’t take an IFormatProvider will default to the current culture. And that’s exactly what happens when we call this method on “meters.” The test above is flawed because it implicitly assumes a culture where the dot is used for decimal separators.
4. Independence
The fourth must-have for unit tests is independence. This one is strongly related to the previous must-have. In fact, you can think of it as a special case of reliability. And what should your unit tests be independent of?
Another unit test.
A unit test should never depend on another test. One test should never do any preparation for the next test, nor should it expect another test to do the same for it.
We can go further and say that this vocabulary is flawed. It doesn’t make sense to talk about a “previous” and “next” test since a real unit test should be completely unaware of the existence of other tests. From its point of view, it is the only test.
And what is all this independence good for? First of all, it adds to the overall reliability of the suite. Let’s say we have tests A, B, and C. A developer makes a change to some code that is exercised by A. Then A starts failing—and rightly so since the developer inadvertently introduced a bug with their changes.
Should B and C fail as well? If they don’t exercise the changed code, they shouldn’t. But let’s say that right after its assertion, A sets up some values that are used by both B and C. Now when A fails, the code after the assertion doesn’t run, the values aren’t set up, and B and C start to fail.
Of course, such an example isn’t even really needed when you think about it. With such a dependency in place, it would be impossible to run B or C on their own without first running A as well.
Creating dependencies between tests is a recipe for disaster. It makes for brittle tests, which require complicated setups and, as a consequence, fail randomly and aren’t run as often as they should be. Keep your tests wholly isolated and independent, both from external dependencies and from one another.
5. Simplicity
Last but not least, I give you the really-must-have of unit testing: simplicity. Be dead simple when writing tests. Resist the temptation to get fancy. You know when you’re writing a test and you hardcode a value and immediately think, “Hmm, it’s so ugly to hardcode this value here. I could generate it on the fly!”
Please, don’t!
That’s precisely the kind of thing that leads to untrustworthy tests. When you try to go the fancy route and generate values instead of hardcoding them, you risk duplicating the implementation code in the test code. And if you’re unlucky enough to have gotten it wrong in both places, you know what’ll happen? The implementation will be wrong, but the test will pass. Which frankly is worse than having no tests at all!
Keep your test code dead simple. No “if” statements. No loops. Just do the good old Arrange-Act-Assert thing, and you’re on the right track.
“But I really need to loop/make a decision in a test. What can I do???”
Do you feel the urge to use an “if” statement in a test? Chances are you should split it into two tests. But if what you really wish for is to loop through a series of values, consider if NUnit’s parametrized tests won’t do the trick.
If nothing helps, then create a utility class, move all the fancy stuff to this new class, and then use it on your tests. It should go without mention that the utility class itself should be tested, right?
Wrapping up
Do you write unit tests? If so, then congratulations! But if the answer is “no,” then please consider starting ASAP. You’ve already seen studies demonstrating the benefits of unit testing. Now we’ve just shown you the qualities a great test must have.
The beginning can be rough, but it’s worth it. The benefits of testing far outweigh the costs. And the sooner you begin, the more you get to practice and the sooner you’ll reap the rewards.
See you next time!
So far I agree with your points, except one thing in your fifth category simplicity. Whenever I need some kind of hard-coded string I’ll almost never write them directly into code (e.g. username, mail address, etc.). Instead I use Guid.NewGuid().ToString() to create some kind of randomized text which will be used (maybe with a format string if a specific format like a email address is needed). This ensures that the underlying business logic really works with arbitrary values (within a specific range) and doesn’t depend on a exact value that is the same as used within the test.
If the simple approach of Guid.NewGuid() isn’t enough, then possibly consider the usage of a library that will produce some fake data like https://www.nuget.org/packages/Bogus/. Yes, I know this could lead to the tests flickering from good to bad and back without any code change, but if this starts to happen you know there is a deeper problem within your code that must be solved asap.