Unit Testing – Alternatives for Unit Testing

testingunit testing

The problem I have with unit testing is that while it makes refactoring(regression testing) easier, it expands the code base and make it more difficult to prototype or to change the design. Developers tend to be averse to change the API due to unit testing.

Is it possible to develop a testing system such that you don't have to write tests?

For an example, it would show the resultant values for every value in the domain ( a model of the function) and the programmer can choose to accept it as the test. That way you can diff the model to find regression bugs unlike property based testing.

Best Answer

This is impossible:

  • Unless you only restrict yourself to pure functions, the result value may not capture everything a subroutine does. Many subroutines have useful side-effects; these side-effects are not captured in the result value. E.g., C's printf returns the number of characters written, but the more interesting feature of printf is that it writes something, and that is not captured in the return value.
  • It is infeasible for large domains to show the result for every value in the domain, e.g. a Java method foo(long, long, long) has 2192 (~1064) different possible inputs, that's close to the number of particles in the universe, and even if you had a 1000000 core CPU with 10GHz that could show one result for every clock cycle, it would take you ~20000000000000000000000000000000000 years. Even worse: most domains are infinite, e.g. there are infinitely many Strings, infinitely many Ruby Integers, etc.
  • It is impossible to determine what the result of a subroutine is without running it. (This is a variation of the Function Problem which is provably undecidable.) Heck, it is even impossible to determine whether it even has a result or not. (This is called the Halting Problem and is provably undecidable.) So, the only way to generate the result values is to run the subroutine for every combination of possible inputs, but you have no idea what the subroutine does! It might format your hard disk, for example. (And you can't figure out what it does, because of the Function Problem.)
Related Topic