I promise that I’ll get to a treatment of code coverage tools in short order.  In this post, I’ll go through 6 different options and list their features to help you make a decision.

But first, I want to offer a quick disclaimer and warning.

If you’re here because you’re looking for a way to let management track the team’s code coverage progress, please stop and reconsider your actions.

I’ve written in the past about how code coverage should not be a management concern, and that holds as true now as ever.  Code coverage is a tool that developers should use — not something that should be done to them.  Full stop.

With that out of the way, let’s get back to helping developers pick a tool to keep an eye on code coverage.

What Is Code Coverage and Why Do It?

Okay, with that out of the way, let’s talk extremely briefly about what code coverage is.

And take note that this is going to be a very simple explanation, glossing over the fact that there are actually (slightly) different ways to measure coverage.  But the short form is this.  Code coverage measurements tell you which lines of code your test suite executes and which lines it doesn’t.

Let’s look at the simplest imaginable example.  Here’s a pretty dubious implementation of division:

Let’s say this was the only method in our codebase.  Let’s also say that we wrote a single unit test, from which we invoked Divide(2, 1) and asserted that we should get back 2.

We would then have 67% code coverage.  Why?

Well, this test would cause the runtime to test the conditional and then to execute the return x/y statement, thus executing two-thirds of the method.  Absent any other tests, no automated test ever executes that last line.

So in the grossest of terms, that’s the “what.”  As for the “why,” developers can use code coverage analysis to see holes in their testing efforts.  For instance, code coverage analysis would tell us here, “Hey, you need to test the case where you pass a 0 to the method for y.”

What Are Code Coverage Tools?

What, then, are code coverage tools?

Well, as you can imagine, computing code coverage the way I just did would get pretty labor intensive.

So developers have done what developers do.  They’ve automated code coverage detection.

In just about any language and tech stack imaginable, you have ways to detect how thoroughly the unit test suite covers your codebase.  (Don’t confuse code coverage with an assessment of the quality of your tests, though.  It’s just a measure of whether or not the tests execute the code.)

The result is an array of tools to choose from that help you see how well your test suite covers your codebase.  And that can become somewhat confusing.  So today, I’ll take you through some of your options in the .NET ecosystem.

Let’s take a look at those options.

1. Default Visual Studio Code Coverage

Before you gear up to start downloading things or breaking out your wallet, understand that you may already have this capability.  It depends on your version of Visual Studio.  If you have the enterprise version, you can take advantage of this capability out of the box.

And it’s as simple as this screenshot.

You can read about this in more detail at the Microsoft documentation site.  But this is a nice, comprehensive option that requires very little additional work, assuming you have the correct version.  And it has cool features:

• It reports coverage percentages at various granularities (e.g., assembly, class, etc.).
• You can select all of your tests or subsets of them.
• You can have it paint your IDE, meaning it lets you actually visualize the coverage as you look at your code.
• It comes from Microsoft, so you can expect plenty of maintenance and support if you invest in its usage.

2. Rider’s and ReSharper’s dotCover

If you have a Visual Studio Enterprise license, your life is good in a lot of ways.  But if you don’t, shelling out $3,000 per year is probably going to be a bit of a deal-breaker just to see code coverage.

And it should be a deal breaker.  Visual Studio Enterprise is an AWESOME tool, but if the only reason you’re buying it is to see code coverage, you have significantly cheaper options.

Take JetBrains dotCover, for instance.  This comes with ReSharper Ultimate, which costs only $299 per year.  So you get coverage analysis and you also get Rider and ReSharper, which are really cool tools.

With dotCover, you get plenty of features as well.  Some highlights include

• Detailed reporting.
• Test runners.
• Both Visual Studio and CI integration of coverage measurements.
• Navigate to covering tests.
• A cool “hotspots” view that calls out risky methods.
• IDE painting.

3. NCrunch

I’m going to switch gears a little now, having given a treatment to some major players in the .NET code coverage tools space.  So far, I’ve listed what I think of as “traditional” code coverage tools.

NCrunch is different.

I mean, it is a code coverage tool, and it will give you code coverage data.  It will also give you the IDE painting that some of the other tools give you, as well.  (If you look at the screenshot of my environment above, you’ll see red and green dots — those are NCrunch telling me that tests are failing and passing, respectively).

But it gives you a killer feature as well.

NCrunch constantly runs your tests, in real-time, as you type.  With NCrunch, you don’t need to run your tests, or even compile, to get feedback on whether your code changes are breaking anything.  The dots will change as you type.  And its pricing is very reasonable, starting at $159 per seat for an individual developer.

Of course, NCrunch has other features besides that killer one, as well:

• Small and customizable memory footprint.
• Parallel test execution.
• Inline exception details and easy VS-integrated debugging.
• Code coverage data and performance metrics.

4. NCover (put in maintenance mode)

So far, we’ve looked at a couple of code coverage tools that come bundled with other products.  So, while they integrate nicely into your development environment, they aren’t exclusively focused on the subject of code coverage.

NCover, however, is solely focused on that.  By specializing, NCover provides extremely detailed information, not only supporting measurements of coverage but also integrating it for the whole team. It tracks trends in coverage and provides detailed reporting.

NCover is a comprehensive tool for this purpose.  Some features include

• Detailed and centralized data about coverage.
• Extensive documentation and user support.
• 32- and 64-bit support, plus memory consumption optimization.
• IDE painting.

Unfortunately by now (2022 / 2023) NCover is in Maintenance mode so don’t expect any new features.

5. OpenCover (put in archive mode)

Unfortunately by now (2022 / 2023) OpenCover is in archive mode and won’t evolve with the new version of .NET.

Since everything requires a bit of an investment so far, it’s worth asking if there are any free tools out there.  And the answer is yes. here’s OpenCover.

OpenCover supports .NET 2 and above (only on Windows) and works with both 32-bit and 64-bit processors, and you can get it via NuGet.  It originally arose out of attempts to grow a tool called PartCover, and it eventually grew into its own tool.

Features include

• 32- and 64-bit support.
• Both branch and statement coverage support (two subtly different ways to measure coverage).
• Generation of an XML-based report that, combined with ReportGenerator, produces a nice HTML-based report on coverage.
• Free and open source, so you can adjust to suit your needs.

6. NDepend

I’ll close out (not surprisingly, on the NDepend blog) by mentioning NDepend. 

While NDepend doesn’t directly measure code coverage, it happily imports the data from other coverage tools and lets you do some really powerful things with it.

NDepend gives you tons of metrics out of the box, and when you combine those with the imported coverage data, you can do important assessments of your codebase, such as assessing method risk.  You can also make use of NDepend’s heat map capabilities to get powerful visuals of where you have good coverage and where you don’t.

But my favorite part, personally, is how you can integrate this data with CQLinq to create your own rules and measurements as they relate to coverage.

NDepend will, for example, tell you by default whether you’ve got full coverage since the last baseline.  But you can easily customize that further to do something like discount all code created by a code generator or all code that inherits from a certain type.

The sky is the limit when it comes to incorporating test coverage data with other metrics about your code.

And that brings us back around to my initial warning in the beginning.

Code coverage isn’t something management should use to take developers to task; rather, it’s something that developers should use to make sure they’re satisfied with their codebases.  Use one or more of these tools to measure your coverage, and then use NDepend to create seriously actionable information on the basis of coverage.

Then you’ll have the sort of clean, reliable codebase that management doesn’t ever worry about.

Erik Dietrich

Website

I'm a passionate software developer and active blogger. Read about me at my site.