Is it significantly costlier to fix a bug at the end of the project

debuggingestimationproject-management

In a blog post by Andrew Hay, the following axiom was posited:

It costs significantly more to fix a bug at the end of the project that it does to fix the same bug earlier in the project.

However, this doesn't seem certain, especially after reading a blog post on Less Wrong, and the data I've seen to back it up is extremely old.

Is this still axiom accurate today?

Best Answer

The only hard data I've ever seen is Boehm and Papaccio, Understanding and Controlling Software Costs.

This stems back to 1988 and was a study of around 80 software projects. They concluded that a decision made early and corrected late could cost 50-200 times what it would have if it had been corrected early. But the kind of very early decisions they're talking about are which OS to run on and which language and database to use.

So those figures may be overwrought with respect to today's software development. However, now we have a lot of experience in the field and we know instinctively that it still holds true to a point.

In the extreme, we know that if a failure in requirements is caught just before we go to production, this causes a lot of rework and get the project delayed or indeed cancelled, where if it had been caught before any work had been done, we'd have been fine.

Edit: Doc Brown makes a good point in his comment.

Boehm's research was done on COBOL and FORTRAN projects in a time when compile and run times were ridiculously slow. I started my career in the early 90s on COBOL and the compile and test cycle used to take so long that it was worth the effort to dry-test the code before going through the cycle (or at least during the build phase, just in case you could catch something and cancel it early, saving yourself an hour or so).

In turn, our bosses used to laugh at our complaints because it wasn't so long ago they used to have to carry a box of hand-sorted punch cards to the server room and leave it there for a day.

So it was definitely more true then than it is now.

And yet, very recently, I've seen blogs reusing Steve McConnell's visualisation of this problem (ref, dated 1996) as if that graph was actually based on hard numbers. It isn't. It's a visualisation, to explain his point simply.

I think that Morendil's premise, in the article the OP quoted, is a good one. The science we have on this subject is poor and outdated and yet treated as canon. But I also think that it holds up well and sounds true because we know from bitter experience that it is still true, at least to a point. And I think his dramatic "diseased discipline" phrasing does him no favours.