NDepend

Improve your .NET code quality with NDepend

How to Measure Lines of Code? Let’s Count the Ways

There are a few ways to count lines of code, and they each have their advantages and disadvantages.

Much of the differences come down to defining what a “line” is. Is a line a literal line in the source file, a logical statement in the language we’re using, or an executable instruction?

Let’s take a look at three metrics:

Source lines of code—the number of lines of code in a method, skipping comments and blank lines
Logical lines of code—the number of statements, ignoring formatting and often counting a line as more than one statement
IL instructions—the number of instructions that the code compiles to

Is one better than the other? It depends on what you’re trying to measure.

Continue reading How to Measure Lines of Code? Let’s Count the Ways

Functional Programming Makes Your Code Not OO…And That’s It

Over the course of the fall and winter, I’ve been gaining momentum with code research posts. Today, I bring that momentum to bear on the subject of functional programming. Or at least, on functional style of programming in C# codebases.

But before I do that, let me provide a little background in case you haven’t caught the previous posts in the series. It started with me doing automated static analysis on 100 codebases to see how singletons impact those codebases. Next up, I used that data to look at how unit tests affect codebases. That post generated a lot of buzz, so I enlisted a partner to help with statistical analysis and then boosted the codebase sample size up to 500.

At the end of that last post, I suggested some future topics of study. Well, now I’ve picked one: functional programming.

What Is Functional Programming?

The idea with this post is mostly to report on findings, but I’d be remiss if I didn’t provide at least some background so that anyone reading has some context. So first, let’s cover the topic of functional programming briefly.

Functional programming is one of the major programming paradigms. Specifically, its calling card is that it disallows side effects. In other words, it models the rules of math, in which the result of the function (or method) is purely a deterministic function of its inputs.

So, in pseudo-code, it looks like this:

public int Add(int x, int y)
{
  return x + y;
}

public int Add(int x, int y)

{

return x + y;

}

This is a functional method. But if you do something like this

public int Add(int x, int y)
{
  return GlobalVariable + x + y;
}

public int Add(int x, int y)

{

return GlobalVariable + x + y;

}

or like this

public int Add(int x, int y)
{
  _databasePlopper.plopResultInDatabase(x + y); 
  return x + y;
}

public int Add(int x, int y)

{

_databasePlopper.plopResultInDatabase(x + y);

return x + y;

}

then you’re out of the functional realm because you’re adding side effects. These two modified versions of Add() each concern themselves with the world beyond processing the inputs to add. (As an aside, you could “fix” this by passing the global variable or the _databasePlopper dependency to the method as a parameter.)

Now, take note of something because this matters to the rest of the post. While C# (or any other object-oriented language) is not a functional language, per se, you can write functional methods in C#.

Continue reading Functional Programming Makes Your Code Not OO…And That’s It

Unit Tests Correlate With Desirable Codebase Properties

Today, I give you the third post in a series about how unit tests affect codebases.

The first one wound up getting a lot of attention, which was fun. In it, I presented some analysis I’d done of about 100 codebases. I had formed hypotheses about how I thought unit tests would affect codebases, and then I tested those hypotheses.

In the second post, I incorporated a lot of the feedback that I had requested in the first post. Specifically, I partnered with someone to do more rigorous statistical analysis on the raw data that I’d found. The result was much more clarity about not only the correlations among code properties but also how much confidence we could have in those relationships. Some had strong relationships while others were likely spurious.

In this post, though, I’m incorporating the single biggest piece of feedback. I’m analyzing more codebases.

Analysis of 500 (ish) C# Codebases

Performing static analysis on and recording information about 500 codebases isn’t especially easy. To facilitate this, I’ve done significant work automating ingestion of codebases:

Enabling autonomous batch operation
Logging which codebases fail and why
Building in redundancy against accidentally analyzing the same codebase twice.
Executing not just builds but also NuGet package restores and other build steps.

That’s been a big help, but there’s still the matter of finding these codebases. To do that, I mined a handful of “awesome codebase” lists, like this one. I pointed the analysis tool at something like 750 codebases, and it naturally filters out any that don’t compile or otherwise have trouble in the automated process.

This left me with 503 valid codebases. That number came down to 495 once adjusted for codebases that, for whatever reason, didn’t have any (non-third party) methods or types or that were otherwise somehow trivial.

So the results here are the results of using NDepend for static analysis on 495 C# codebases.

Continue reading Unit Tests Correlate With Desirable Codebase Properties

The Unit Test Effect Study, Refined

About a month ago, I wrote a post about how unit tests affect (and apparently don’t affect) codebases. That post turned out to be quite popular, which is exciting. You folks gave a lot of great feedback about where we might next take the study. I’ve incorporated some of that feedback and have a followup on the unit test effect on codebases.

Summarized briefly, here are the high points of this second installment and refinement of the study:

Eliminating the “buckets” from the last time.
Introducing more statistical rigor.
Qualifying and refining conclusions from last time.

Also, for the purposes of this post, please keep in mind that non-incorporation of feedback is not a rejection of that feedback. I plan to continue refinement but also to keep posting about progress.

Addressing Some of the Easier Questions and Requests

Before getting started, I’ll answer a few of the quicker-to-answer items that arose out of the comments.

Did your analysis count unit test methods when assessing cyclomatic complexity, etc.?

Yes. It might be interesting to discount unit test methods and re-run analysis, and I may do that at some point.

Can you show the code you’re using? Which codebases did you use?

The scraping/analysis tooling I’ve built using the NDepend API is something that I use in my consulting practice and is in a private repo. As for the list of specific codebases, I’m thinking I’ll publish that following the larger sample size study. In the most general terms, I’m going through pages like this that list (mostly) C# repos and using their links.

What about different/better categorization of unit test quality (test coverage, bolted on later vs. written throughout vs. demonstrably test driven)?

This is definitely something I want to address, but the main barrier here is how non-trivial this is to assess from a data-gathering perspective. So I will do this, but it will also take time.

Think of even just the anecdotally “easy” problem of determining TDD vs. non-TDD. I approximated this by positing that test-driving will create a certain ratio of test methods to production methods since any production method will be preceded by a test method (notwithstanding future extract method refactorings). We could, perhaps, do better by auditing source control history and looking for a certain commit cadence (modification to equal numbers of test/production classes, for instance). But that’s hard, and it doesn’t account for situations with large batch commits, for instance.

The upshot is that it’s going to take some doing, but I think we collectively can figure it out.

Continue reading The Unit Test Effect Study, Refined

Lack of Cohesion of Methods: What Is This And Why Should You Care?

Lack of cohesion of methods (sometimes abbreviated LCOM) is one of those things that occurs fairly high up on the software hierarchy of needs. What’s the “software hierarchy of needs?” It’s a thing that I just made up, shamelessly copying Maslow’s Hierarchy of Needs. (Though in a quick search to see how effectively I’ve planted this flag, I see that Scott Hanselman had the idea before me. But mine looks a little different than his because I’m talking about the software, rather than the humans writing it.)

Anyway, here’s the hierarchy.

The cost of change stays relatively flat because the software is highly maintainable.
You can change it over time in response to evolving requirements and market realities.
It performs and behaves acceptably in production.
Technically, it fulfills the functional requirements and user value proposition.
It simply exists in production.

When it comes to something like lack of cohesion of methods, you’re talking about the top two categories. Early in your career, you’ll tend to have worries like just wheezing past the finish line with a feature and like getting your code to do what you want it to. With practice and time, you then start worrying about non-functional concerns and maintainability. And when you start worrying about that, you should start paying attention to lack of cohesion of methods (among other things).

So for the rest of this post, I’ll focus on this one idea: lack of cohesion of methods. What is it, how do you measure it, and why should you care?

Continue reading Lack of Cohesion of Methods: What Is This And Why Should You Care?

Unit testing doesn't affect codebases the way you think.

Unit Testing Doesn’t Affect Codebases the Way You Would Think

I’ve just wrapped up another study. (The last one was about singletons, if you’re interested.) This time, I looked at unit testing and the impact it has on codebases.

It didn’t turn out the way I expected.

I’m willing to bet that it also won’t turn out that you expected these results. It had a profound effect on certain unexpected codebase properties while having minimal effect on some that seem like no-brainers. But I’ll get to the specifics of that shortly.

First, let me explain a bit about expectations and methodology.

Unit Testing: The Expected Effect on Code

Let’s be clear for a moment here. I’m not talking about the expected effect of unit tests on outcomes. You might say, “We believe in unit testing because it reduces defects” or, “We unit test to document our design intentions,” and I didn’t (and probably can’t) measure those things.

When I talk about the effect of unit testing, I’m talking about the effect it has on the codebase itself. Let’s consider a concrete example to make it clear what I mean. You can’t (without a lot of chicanery) unit test private methods, and you can’t easily unit test internal methods. This creates a natural incentive to make more methods public, so we might expect heavily unit-tested codebases to feature more public methods.

This actually turns out to be true.

I’ll talk more about the axes later in the post. But for now, check out the plot and the trend line. More unit testing means a higher percentage of public methods. Score one for that hypothesis!

With a win there, let’s think of some other hypotheses that seem plausible. Methods with more “going on” tend to be harder to test. So you’d expect relatively simple methods in a highly tested codebase. To get specific, here was what I anticipated in heavily tested codebases:

Less cyclomatic complexity per method.
Fewer lines of code per method.
Lower nesting depth.
Fewer method overloads.
Fewer parameters.

I also had some thoughts about the impact on types:

More interfaces (this makes testing easier).
Less inheritance (makes testing harder).
More cohesion.
Fewer lines of code.
Fewer comments.

Continue reading Unit Testing Doesn’t Affect Codebases the Way You Would Think

CRAP Metric Is a Thing And It Tells You About Risk in Your Code

I won’t lie. As I thought about writing this post, I took special glee in contemplating the title. How should I talk about the CRAP metric? *Snicker*

I guess that just goes to show you that some people, like me, never truly grow up. Of course, I’m in good company with this, since the original authors of the metric had the same thought. They wanted to put some quantification behind the common, subjective declaration, “This code is crap!”

To understand that quantification, let’s first consider that CRAP is actually an acronym: C.R.A.P. It stands for change risk anti-patterns, so it addresses the risk of changing a bit of code. In other words, methods with high CRAP scores are risky to change. So the CRAP metric is all about assessing risk.

When you get a firm grasp on this metric, you get a nice way to assess risk in your codebase.

The CRAP Metric: Getting Specific

Okay, so how does one quantify risk of change? After all, there are a lot of ways that one could do this. Well, let’s take a look at the formula first. The CRAP score is a function of methods, so we’ll call it CRAP(m), mathematically speaking. (And yes, typing CRAP(m) made me snicker all over again.)

Let CC(m) = cyclomatic complexity of a method and U(m) = the percentage of a method not covered by unit tests.

CRAP(m) = CC(m)^2 * U(m)^3 + CC(m).

Alright, let’s unpack this a bit. To arrive at a CRAP score, we need a method’s cyclomatic complexity and its code coverage (or really lack thereof). With those figures, we multiply the square of a method’s complexity by the cube of its rate of uncovered code. We then add its cyclomatic complexity to that. I’ll discuss the why of that a little later, but first let’s look at some examples.

First, consider the simplest, least CRAP-y method imaginable: a method completely covered by tests and with no control flow logic. That method has a cyclomatic complexity of 1 and uncovered percentage of 0. That means that CRAP(m) = 1^2 * 0^3 + 1 = 1. So the minimum CRAP metric score is 1. What about a gnarly method with no test coverage and cyclomatic complexity of 6? CRAP(m) = 6^2 * 1^3 + 1 = 37.

The authors of this metric define a CRAP-y method as one with a CRAP score greater than 30, so that last method would qualify.

Continue reading CRAP Metric Is a Thing And It Tells You About Risk in Your Code

The Singleton Design Pattern: Impact Quantified

This post has been about a month in the offing. Back in August, I wrote about what the singleton pattern costs you. This prompted a good bit of discussion, most of which was (as it always is) anecdotal. So a month ago, I conceived of an experiment that I called the singleton challenge. Well, the results are in. I’m going to quantify the impact of the singleton design pattern on codebases.

I would like to offer an up-front caveat. I’ve been listening lately to a fascinating audiobook called “How to Measure Anything,” and it has some wisdom for this situation. Measurement is primarily about reducing uncertainty. And one of the driving lessons of the book is that you can measure things — reduce uncertainty — without getting published in a scientific journal.

I mention that because it’s what I’ve done here. I’ll get into my methodology momentarily, but I’ll start by conceding the fact that I didn’t (and couldn’t) control for all variables. I looked for correlation as a starting point because going for causation might prove prohibitive. But I think I took a much bigger bite out of trying to quantify this than anyone has so far. If they have, I’ve never seen it.

A Quick Overview of the Methodology

As I’ve mentioned in the past on this blog, I earn a decent chunk of my consulting income doing application portfolio assessments. I live and breathe static code analysis. So over the years, I’ve developed an arsenal of techniques and intellectual property.

This IP includes an extensive codebase assessor that makes use of the NDepend API to analyze codebases en masse, store the results, and report on them. So I took this thing and pointed it at GitHub. I then stored information about a lot of codebases.

But let’s get specific. Here’s a series of quick-hitter bullets about the experiment that I ran:

I found this page with links to tons of C# projects on GitHub, so I used that as a “random” selection of codebases that I could analyze.
I gave my mass analyzer an ordered list of the codebase URLs and turned it loose.
Anything that didn’t download properly, decompress properly, or compile properly (migrating for Core, restoring NuGet packages, and building from command line) I discarded. This probably actually creates a bias toward better codebases.
Minus problematic codebases, I built all solutions in the directory structure and made use of all compiled, non-third-party DLLs for analysis.
I stored the results in my database and queried the same for the results in the rest of the post.

I should also note that, while I invited anyone to run analysis on their own code, nobody took me up on it. (By all means, still do it, if you like.)

Singleton Design Pattern: the Results In Broad Strokes

First, let’s look at the scope of the experiment in terms of the code I crunched. I analyzed

100 codebases
986 assemblies
5,086 namespaces
72,615 types
501,257 methods
1,495,003 lines of code

From there, I filtered down raw numbers a bit. I won’t go into all of the details because that would make this an immensely long post. But suffice it to say that I discounted certain pieces of code, such as compiler-generated methods, default constructors, etc. I adjusted this so we’d look exclusively at code that developers on these projects wrote.

Now, let’s look at some statistics regarding the singleton design pattern in these codebases. NDepend has functionality for detecting singletons, which I used. I also used more of its functionality to distinguish between stateless singleton implementations and ones containing mutable state. Here’s how that breaks down:

Continue reading The Singleton Design Pattern: Impact Quantified

Get Smart -- Go Beyond Cyclomatic Complexity C#

Get Smart — Go Beyond Cyclomatic Complexity in C#

Recently, I wrote a post explaining the basics of cyclomatic complexity. You can read that for a deep dive, but for our purposes here, let’s be brief about defining it. Cyclomatic complexity refers to the number of “linearly independent” paths through a chunk of code, such as a method. Understand this by thinking in terms of debugging. If you could trace only one path through a method, it has a cyclomatic complexity of one. But throw a conditional in there, introducing a second path you could trace, and the complexity grows to two.

Today, I’ll talk specifically about C#. Cyclomatic complexity in C# is just, well, cyclomatic complexity applied to the language C#. No big mystery there.

But what I want to talk about today is not cyclomatic complexity — not per se. Today, I’d like to talk about how you can go beyond cyclomatic complexity in C# to get some even more intelligent metrics. How can you really zero in on sources of risk and complexity in your code?

Wait, What’s Wrong with Cyclomatic Complexity?

Let me be clear. There’s absolutely nothing wrong with tracking cyclomatic complexity.

It’s a great aid and shorthand for reasoning about your code’s relative complexity and for understanding where testing challenges lie. You can use it to locate complexity “hot spots” in your code and then to address them in ways that make sense. So no criticism whatsoever. I’m just advocating that you go beyond it.

Think of it this way. When I encourage you to install a Visual Studio plugin, I’m not knocking Visual Studio. Visual Studio is a wonderful and productive IDE, in my estimation. Instead, I’m encouraging you to make it even better — to enhance your experience.

The same sort of reasoning applies here. Cyclomatic complexity is a great start for reasoning about your code’s complexity. But we can add some considerations to make your life even better. Let’s take a look at those.

The rest of this post will show you in detail what that looks like. But if you want to try it out for yourself, you’ll need to download a copy of NDepend.

Continue reading Get Smart — Go Beyond Cyclomatic Complexity in C#

Understanding Cyclomatic Complexity

Wander the halls of an enterprise software outfit looking to improve, and you’ll hear certain things. First and foremost, you’ll probably hear about unit test coverage. But, beyond that, you’ll hear discussion of a smattering of other metrics, including cyclomatic complexity.

It’s actually sort of funny. I mean, I understand why this happens, but hearing middle managers say “test coverage” and “cyclomatic complexity” has the same jarring effect as hearing developers spout business-meeting-speak. It’s just not what you’d naturally expect.

And you wouldn’t expect it for good reason. As I’ve argued in the past, code coverage shouldn’t be a management concern. Nor should cyclomatic complexity. These are shop-heavy specifics about particular code properties. If management needs to micromanage at this level of granularity, you have a systemic problem. You should worry about these properties of your code so that no one else has to.

With that in mind, I’d like to focus specifically on cyclomatic complexity today. You’ve probably heard this term before. You may even be able to rattle off a definition. But let’s take a look in great detail to avoid misconceptions and clear up any hazy areas.

Defining Cyclomatic Complexity

First of all, let’s get a specific working definition. This is actually surprisingly difficult because not all sources agree on the exact method for computing it.

How can that be? Well, the term was dreamed up by a man named Thomas McCabe back in 1976. He wanted a way to measure “the number of linearly independent paths through a program’s source code.” But beyond that, he didn’t specify the mechanics exactly, leaving that instead to implementers of the metric.

He did, however, give it an intimidating-sounding name. I mean, complexity makes sense, but what does “cyclomatic” mean, exactly? Well, “cyclomatic number” serves as an alias for something more commonly called circuit rank. Circuit rank measures the number of independent cycles within a cyclic graph. So I suppose he coined the neologism “cyclomatic complexity” by borrowing a relatively obscure discrete math concept for path independence and applying it to code complexity.

Well then. Now we have cyclomatic complexity, demystified as a term. Let’s get our hands dirty with examples and implications.

Continue reading Understanding Cyclomatic Complexity