NDepend

Improve your .NET code quality with NDepend

Should You Aim for 100 Percent Test Coverage?

Test coverage serves as one of the great lightning rods in the world of software development.  First, people ask whether it makes for a good metric at all.  Then they ask, if you want to use it as a metric, should you go for 100 percent coverage?  If not, what percentage should you go for? Maybe 42 percent, since that’s the meaning of life?

I don’t mean to trivialize an important discussion.  But sometimes it strikes me that this one could use some trivializing.  People dig in and draw battle lines over it, and counterproductive arguments often ensue.  It’s strange how fixated people get on this.

I’ll provide my take on the matter here, after a while.  But first, I’d like to offer a somewhat more philosophical look at the issue (hopefully without delving into overly abstract navel-gazing along the lines of “What even is a test, anyway, in the greater scheme of life?”)

What Does “Test Coverage” Measure?

First of all, let’s be very clear about what this metric measures.  Many in the debate — particularly those on the “less is more” side of it — quickly point out that test coverage does not measure the quality of the tests.  “You can have 100 percent coverage with completely worthless tests,” they’ll point out.  And they’ll be completely right.

To someone casually consuming this metric, the percentage can easily mislead.  After all, 100 percent coverage sounds an awful lot like 100 percent certainty.  If you hired me to do some work on your car and I told you that I’d done my work “with 100 percent coverage,” what would you assume?  I’m guessing you’d assume that I was 100 percent certain nothing would go wrong and that I invited you to be equally certain.  Critics of the total coverage school of thought point to this misunderstanding as a reason not to pursue that level of test coverage.  But personally, I just think it’s a reason to clarify definitions.

Continue reading Should You Aim for 100 Percent Test Coverage?

In Defense of Using Your Users as Software Testers

In most shops of any size, you’ll find a person that’s just a little too cynical.  I’m a little cynical myself, and we programmers tend to skew that way.  But this guy takes it one step further, often disparaging the company in ways that you think must be career-limiting.  And they probably are, but that’s his problem.

Think hard, and some man or woman you’ve worked with will come to mind.  Picture the person.  Let’s call him Cynical Chad. Now, imagine Chad saying, “Testing? That’s what our users are for!”  You’ve definitely heard someone say this at least once in your career.

This is an oh-so-clever way to imply that the company serially skimps on quality.  Maybe they’re always running behind a too-ambitious schedule.  Or perhaps they don’t like to spend the money on testing.  I’m sure Chad would be happy to regale you with tales of project manager and QA incompetence.  He’ll probably tell you about your own incompetence too, if you get a couple of beers in him.

But behind Chad’s casual maligning of your company lies a real phenomenon.  With their backs against the wall, companies will toss things into production, hope for the best, and rely on users to find defects.  If this didn’t happen with some regularity in the industry, it wouldn’t be fodder for Chad’s predictable jokes and complaints.

The Height of Unprofessionalism

Let’s now forget Chad.  He’s probably off somewhere telling everyone how clueless the VPs are, anyway.

Most of the groups that you’ll work with as a software pro would recoil in horror at a deliberate strategy of using your users as testers.  They work for months or years implementing the initial release and then subsequent features.  The company spends millions on their salaries and on the software.  So to toss it to the users and say “you find our mistakes” marks the height of unprofessionalism.  It’s sloppy.

Your pride and your organization’s professional reputation call for something else.  You build the software carefully, testing as you go.  You put it through the paces, not just with unit and acceptance tests, but with a whole suite of smoke tests, load tests, stress tests and endurance tests.  QA does exploratory testing.  And then, with all of that complete, you test it all again.

Only after all of this do you release it to the wild, hoping that defects will be rare.  The users receive a polished product of which you can be proud — not a rough draft to help you sort through.

Users as Testers Reconsidered

But before we simply accept that as the right answer and move on, let’s revisit the nature of these groups.  As I mentioned, the company spends millions of dollars building this software.  This involves hiring a team of experienced and proud professionals, among other things.  Significant time, money, and company stake go into this effort.

If you earn a living as a salaried software developer, your career will involve moving from one group like this to another.   In each of these situations, anything short of shipping a polished product smacks of failure.  And in each of these situations, you’ll encounter a Chad, accusing the company of just such a failure.

But what about other situations?  Should enlisting users as testers always mean a failure of due diligence?  Well, no, I would argue.  Sometimes it’s a perfectly sound business or life decision.

Continue reading In Defense of Using Your Users as Software Testers

How to Use NDepend’s Trend Charts

Imagine a scene for a moment.  A year earlier, a corporate VP spun up a major software project for his organization.  He brought a slew of his organization’s software developers into the project.  But he also needed to add more staff in the form of contractors.

This strained the budget, so he cut a few corners in terms of team member experience.  The VP reasoned that he could make up for this with strategic use of experienced architects up front.  Those architects would prototype good patterns and make it so the less seasoned contractors could just kind of paint by numbers.  The architects spent a few months doing just that and handed the work off to the contractors.

Fast forward to the present.  Now a consultant sits in a nice office, explaining to a beleaguered VP how they got so far behind schedule.  I can picture this scene quite easily because organizations hire me to be this consultant.  I live this scene over and over again.
Continue reading How to Use NDepend’s Trend Charts

Fixing Your Tangled Dependency Graph

I’ve written before about making use of NDepend’s dependency graph.  Well, indirectly, anyway.  In that post, I talked about the phenomenon of actual software architecture not matching the pretty diagrams people draw in Visio.  It reminds me of Helmuth von Moltke’s wisdom that no battle plan survives contact with the enemy.

Typically, architects conceive of wondrous, clean, and decoupled systems.  Then they immortalize this pristine architecture in Visio.  Naturally, print outs go up on the wall, and everyone knows what the system should look like.  But somehow, it never actually winds up looking like that. Continue reading Fixing Your Tangled Dependency Graph