Monthly Archives: December 2007

The Mythical 5%

Bruce Eckel, author of Thinking in C++, relays a commencement address he gave.

First, some bad news:

The statistics are sobering: 50-80% of programming projects fail. These numbers are so broad because people don’t brag about their failures, so we have to guess. In any event, this makes the world sound pretty unreliable. …

Many projects that do somehow “succeed” still resemble sausage: you don’t want to see how it was made, or what’s in it.

Now, some good news:

An even more fascinating metric is this: 5% of programmers are 20x more productive than the other 95%. If this were a science, like it claims, we could figure out how to get everyone to the same level.

Differences this drastic do exist. Talent, experience and education provide raw material. But it must be cultivated.

So how do you become one of these mythical 5%?

These people are not those who can remember all the moves and have fingers that fly over the keyboard erupting system commands. In my experience those in the 5% must struggle to get there, and struggle to stay there, and it’s the process of continuous learning that makes the difference.

Because of what I do, I’ve met more than my share of these people. They read a lot, and are always ready to tackle a new concept if it looks worthwhile. I think if they do go to conferences they’re very selective about it. Most of their time is spent being productive, figuring things out.

The big issue is knowing that you’re going after that 20x productivity increase. Which means getting leverage on everything you do. Never just “bashing something out,” but using the best tools, techniques, and ideas at your disposal. Always doing your best.

…Being able to analyze and understand a situation and discover the hinge points of a problem is essential; this takes a clear mind and detached perspective. For example, sometimes the choice of programming language makes a huge difference, but often, it’s relatively unimportant. Regardless, people will still spend all their time on one decision while something else might actually have a far greater influence. Architectural decisions, for example.

Well put. Understand the problem well. Avoid the temptation to shoe-horn what seemed to work last time. Think, too: what would you do differently next time? How would you solve the problem in a more clear manner? How would you make the source code–every line–more clear?

Often, you need to “bash[] something out.” Do it in the smartest way possible, despite the pressures to deliver. Hesitating to dive in at the appropriate time is called analysis paralysis: you might need a very imperfect first cut to get past that.

One factor of the 20x difference is learning what you can live with, and fixing what you can’t. The cost of rewriting (let’s say “refactoring”) a piece of code can be painful, but there’s also a cost of living with it. It can be an agonizing call.

Gerald Weinberg … is most famous for saying “no matter what they tell you, it’s always a people problem.”

Usually the things that make or break a project are process and people issues. The way that you work on a day-to-day basis. Who your architects are, who your managers are, and who you are working with on the programming team.

An appropriate finish:

You’ll need to make a lot of mistakes in order to figure things out. So be humble, and keep asking questions.

CastleCops Program List

I get a strange message about a program crashing. I don’t recognize its name (swdsvc.exe). Is it legit or not? When I google it, I don’t know if I should trust the web-sites that come up any more than the mysterious executable. What to do?

CastleCops keeps very valuable lists of programs and things you’d expect to find on your PC, and those you don’t.

Hint: Use the search box. There are a lot of entries.

(As it turns out, swdsvc.exe is SpyWareDoctor.)

Update 1/26/2009: Castlecops’ useful information is now at

Automating Embarrassment

I recently received this memo:

…unbeknownst to us as it only affected certain files and only a few [investors] … our data was unexplainably fatally contaminated. We have worked diligently, along with our software vendor, to fully correct the problem.

I can’t read that without feeling the sting of being in their shoes.

Whose fault was it, the software vendor or the user? Does it matter? This software vendor’s client bears the brunt of this embarrassment with their clients. That’s everybody’s problem.

I won’t name names: I’m not out to further embarrass them.

My point isn’t to gawk, but hear the clarion call to vigilance:

  • The design must be sound.
  • The implementation must be bullet-proof.
  • The code must be as clear as possible.
  • No undefined behavior.
  • No mysteries.
  • Find a problem? (Even with the design?) Fix it. Now.
  • Are users doing things you don’t expect? Resolve it.
  • Do your tests cover what could really happen?

The pressure to deliver is constant, but has to be weighed against what you’re really delivering.

One of modern computing’s founding fathers says:

Computing’s central challenge, viz. “How not to make a mess of it,” has not been met.

—Edsger Dijkstra (November, 2000)