Testing – Does Path Coverage Guarantee Finding All Bugs?

test-coveragetesting

If every path through a program is tested, does that guarantee finding all bugs?

If not, why not? How could you go through every possible combination of program flow and not find the problem if one exists?

I hesitate to suggest that "all bugs" can be found, but maybe that is because path coverage isn't practical (as it is combinatorial) so it isn't ever experienced?

Note: this article gives a quick summary of coverage types as I think about them.

Best Answer

If every path through a program is tested, does that guarantee finding all bugs?

No

If not, why not? How could you go through every possible combination of program flow and not find the problem if one exists?

Because even if you test all possible paths, you still haven't tested them with all possible values or all possible combinations of values. For example (pseudocode):

def Add(x as Int32, y as Int32) as Int32:
   return x + y

Test.Assert(Add(2, 2) == 4) //100% test coverage
Add(MAXINT, 5) //Throws an exception, despite 100% test coverage

It is now two decades since it was pointed out that program testing may convincingly demonstrate the presence of bugs, but can never demonstrate their absence. After quoting this well-publicized remark devoutly, the software engineer returns to the order of the day and continues to refine his testing strategies, just like the alchemist of yore, who continued to refine his chrysocosmic purifications.

-- E. W. Dijkstra (Emphasis added. Written in 1988. It's been considerably more than 2 decades now.)

Related Topic