Monday, February 19, 2007

Why Specifying Software Never Happens

In my last post I asked why is it so hard to specify a computer program before you start writing code? I mentioned how architects can draw blueprints for an entire building without laying a brick, and how scientists can design nuclear weapons without testing them. However computer programmers have huge difficulty grasping what needs to be done before they do it.

There are several reasons why programmers cannot specify their work early on in the process, however most of the reasons are hand-wavy and are part of the same old story we've been hearing for years. So here are the hand-wavy (but important) explanations first, then we'll get on to the real meat of the issue.
  1. Poor Internal Communication
    This is obvious but is something that consultants love to point out so they can charge $300 an hour. The main problem here is that if a developer/customer cannot properly communicate what he wants, then he'll get something else altogether.
    Every organization has communication problems. Ideally we should wire all of our brains together so everyone knows what everyone else is thinking every instant. Until that happens we'll over pay consultants to tell us how to "refactor the enterprise". Besides, there is nothing specific to computer programming here.
  2. Lack of precedent
    Generally whenever a team creates a computer program it is piece of software that does something that no other piece of software has done before. If there was another program which did the same thing, then there would be little reason to reinvent something that already works. Everyone has heard of the not invented here syndrome, but in practice, things get reused if they meet quality standards, are not prohibitively expensive, and have the appropriate software license. Sure there are me too products out there, but most of those products were developed because they didn't meet one of the three above requirements.
    Whenever anything is created without much precedent, it isn't easy to figure out how it should work. When making a building architects already understand all of the basic mechanics of what is physically possible. Scientists already understand the reactions necessary for a nuclear explosion. However if a computer programmer is writing a brand-new piece of code, there is no one telling him how it will work best. Of course as developers we always try to make use of precedent by reusing libraries and sharing code. However at some point you're going to be writing some code that no one has written before.
  3. Complexity
    This is everyone's favorite excuse for failing projects, but really its horse shit. It like a kid complaining "I can't do my homework cause its too hard." Software development is hard. Complexity is a bad excuse why projects fail not a reason. Complexity will provide a good segway into my next and final reason why systems cannot be specified earlier in the development process.
My so-called hand-wavy explanations don't explain the full problem. Communication issues can be solved with proper communication and management. Lack of precendent and complexity are complaints more than reasons that projects fail.

I believe that there is one major reason why software engineers do not and cannot specify their products. It is the same reason that the Waterfall Model is broken and also the reason why most software projects end in failure. The reason will be the main focus of this blog and I'll tell you now that it isn't agile programming (though I have nothing against agile)...

No comments: