Unit Testing – When and Why to Use Parameterized Tests

testingunit testing

Recently at work we've been having some differences of opinion with regard to Parameterized testing. Normally we use a TDD-style (or at least try to) so I understand the benefits of that approac. However, I'm struggling to see the gain parameterized tests bring. For reference, we work on a service and it's libraries which are exposed via a RESTful interface.

What I've seen so far is tests that are, at least using JUnit within Eclipse:

  • Lacking in detail – when a test fails it's very hard to see the parameters which caused it to fail
  • Often complicated to create
  • Tend to be created after the code has been written – strictly not a drawback as such but do people set out with parameterized tests in mind when they start a piece of code?

If anyone has any examples of where they are really useful or even any good hints for using them that would be fantastic. I want to make sure I'm not just being obstinate because I personally don't choose to use them and see whether they are something we should consider being part of our testing arsenal.

Best Answer

The problem with testing any piece of software is that complexity blows up pretty quickly. The fact is, you can't test all possible combinations of parameters passed to your methods. Phadke advocates a Design of Experiments (DOE) approach, which allows generation of the likely list of parameter values that need to be tested.

The idea is that, even though you are not testing exhaustively, most defects cause a "fault region" to occur rather than an isolated point fault. The DOE approach Phadke advocates using orthogonal arrays samples the parameter space finely enough to hit all possible fault regions.

Isolated faults will probably not be identified, but these are generally fewer than fault regions.

The DOE approach gives you a systematic way of choosing the parameter values to vary.