Managing to Avoid Cobras

Incentives are a funny thing.  Done well, they can spur your team to productivity and career-advancing, win-win situations.  Done not so well, they can create havoc.
As this point, I could offer up a dry explanation of the “law of unintended consequences,” but I’ll let Wikipedia do that.  Instead, I’ll tell you a quick story about something that came to be known as the “cobra effect.”  Snakes are clearly more interesting than laws.

In colonial India, the Brits in charge didn’t like the large number of venomous snakes that they encountered. So they did something that seems imminently sensible at first blush: they offered a bounty to locals for dead cobras.  At first, things went well.  Locals killed corbras, locals got their rewards, and the governing Brits saw a dip in cobra encounters.  But then the cobra encounters started to creep back up, even as the number of dead cobras turned in continued to climb.

Turns out, the locals had started breeding cobras specifically to turn them in and collect the reward.  Once the British government caught wind of this, they (predictably) terminated the reward program, and the erstwhile cobra breeders simply released the now-useless snakes. So the net result of this program turned out to be an increase in the cobras.  Ouch.

Beware of Cobras

When I tell you to beware of cobras, I’m not talking about looking out for snakes (though, clearly, if you live in an area with cobras, you should probably look out for them).  I’m talking about keeping an eye out for bad incentives.  For instance, if you’re managing a software department, do a thought exercise as to what might happen if you agreed to pay testers $5 extra for each bug they found and developers $5 extra for each bug they fixed.  At first, that probably seems like a good way to get them all to work a little extra and produce good results.  But, channeling the cobra example, can you spot what might go wrong?

Managing software developers is hard.  You’re unlikely to understand well what they’re doing on a day-to-day basis since you’re probably no longer as technical as they are. And yet, not only do you have to get the most out of them, but you also have to evaluate them and incentivize them toward good outcomes.

A lot of managers latch onto things like test coverage as an evaluation and incentive.  I’ve talked previously about the problem with that in terms of misleading information, but there’s a secondary problem besides just being wrong about the quality of the codebase.  I’ve heard it said that if you introduce a metric in your organization by which you measure people’s performance, they will meet the goal — even if they have to burn the company to the ground in order to do it.

Put more mildly, when you start measuring people on something like test coverage, high test coverage becomes their goal even though code quality may be your goal.  This means that if it’s more expedient for them to sacrifice quality to get their test coverage up, they’ll do it.  Suddenly, you’ve got a bunch of cobra-breeders on your hands.

What You Can Do

To understand how to avoid this trap, you need to understand why things go off the rails.  And, I don’t just mean, “they created a bad incentive.”  It goes deeper than that.

The problem with these approaches to incentives is that the incenters decided both what they wanted and also how it would be accomplished by others.  Code quality would be improved (what) by writing tests with high coverage (how).  Snake population would be reduced (what) by locals presenting snake carcasses (how).  In both cases, the incenter’s actual goal is obscured by a proxy.  They’re not saying “I want the code to be better” and “I want the snake population reduced,” but rather “I want higher test coverage” and “I want snake carcasses.”

The first step toward creating incentive models that work is to align everyone with with the actual goal, not a proxy.  This can be daunting because we often create proxies when progress toward the actual goal is hard to measure.  It’s hard to know instantaneously how many snakes are around or whether code is “high quality.”

But daunting doesn’t mean impossible.  “High quality code” may be a nebulous concept, but you can get specific about how you define it.  Generally, anyone managing a software development effort is after “high quality” as a vehicle for staying on schedule, minimizing maintenance cost, and eliminating escaped defects.  And so certain questions start to emerge about the code base.

• Is this code easy to read and understand?
• If we modify the code, are we confident that our changes won’t break anything?
• Is the code snarled and coupled, or can you edit one thing without breaking another?
• Do we have proof ahead of compile time that the code does what we believe it does (i.e., automated acceptance tests)?
• Does this code have similar characteristics to known, high quality codebases?

I could probably go on, but you get the idea.  These questions are less daunting to answer, and we can start to conduct automated assessments of code bases that answer them.

Doing this has two beneficial effects.  First, it aligns measurement more closely with your actual goals, making it more likely that the easiest way to earn the reward is to get you what you actually want.  Second, by introducing a lot of measures, it becomes a lot harder to play games, even if participants were so inclined.  It’s a lot harder to game five or ten measures than just one.

So if you want to keep your finger on the pulse of your team’s code quality, there’s work to be done, and there aren’t quick fixes.  You need to invest upfront in tooling and expertise to help rigorously define what quality means.  And then you need to use that same tooling and expertise to create incentives that make sense.  It may seem like overkill, but it’s what is going to prevent you from drowning in cobras.

Picture Courtesy of Bernard Dupont