C++ – Does unit testing lead to premature generalization (specifically in the context of C++)

cgeneralizationtddunit testing

Preliminary notes

I'll not go into the distinction of the different kinds of test there are, there are already a few questions on these sites regarding that.

I'll take what's there and that says: unit testing in the sense of "testing the smallest isolatable unit of an application" from which this question actually derives

The isolation problem

What is the smallest isolatable unit of a program. Well, as I see it, it (highly?) depends on what language you are coding in.

Micheal Feathers talks about the concept of a seam: [WEwLC, p31]

A seam is a place where you can alter behavior in your program without editing in that place.

And without going into the details, I understand a seam — in the context of unit testing — to be a place in a program where your "test" can interface with your "unit".

Examples

Unit test — especially in C++ — require from the code under test to add more seams that would be strictly called for for a given problem.

Example:

  • Adding a virtual interface where non-virtual implementation would have been sufficient
  • Splitting — generalizing(?) — a (smallish) class further "just" to facilitate adding a test.
  • Splitting a single-executable project into seemingly "independent" libs, "just" to facilitate compiling them independently for the tests.

The question

I'll try a few versions that hopefully ask about the same point:

  • Is the way that Unit Tests require one to structure an application's code "only" beneficial for the unit tests or is it actually beneficial to the applications structure.
  • Is the code generalization that is needed to make it unit-testable useful for anything but the unit tests?
  • Does adding unit tests force one to generalize unnecessarily?
  • Is the shape unit tests force on code "always" also a good shape for the code in general as seen from the problem domain?

I remember a rule of thumb that said don't generalize until you need to / until there's a second place that uses the code. With Unit Tests, there's always a second place that uses the code — namely the unit test. So is this reason enough to generalize?

Best Answer

Unit test -- especially in C++ -- require from the code under test to add more seams that would be strictly called for for a given problem.

Only if you don't consider testing an integral part of problem solving. For any nontrivial problem, it ought to be, not only in the software world.

In the hardware world, this has been learnt long ago - in the hard way. The manufacturers of various equipment have learnt through centuries from countless falling bridges, exploding cars, smoking CPUs etc. etc. what we are learning now in the software world. All of them build "extra seams" into their products in order to make them testable. Most new cars nowadays feature diagnostic ports for the repairmen to get data about what's going on inside the engine. A significant portion of the transistors on every CPU serve diagnostic purposes. In the hardware world, every bit of "extra" stuff costs, and when a product is manufactured by the millions, these costs surely add up to large sums of money. Still, the manufacturers are willing to spend all this money for testability. Probably they have figured (or learnt the hard way) that the risk of their product failing in the hands (or under the bums) of end users, and the ensuing loss of reputation, lawsuits, bad PR etc. etc. etc. is way higher than the cost of designing and making their product testable from the start.

Back to the software world, C++ is indeed more difficult to unit test than later languages featuring dynamic classloading, reflection etc. Still, most of the issues can be at least mitigated. In the one C++ project where I used unit tests so far, we didn't run the tests as often as we would in e.g. a Java project - but still they were part of our CI build, and we found them useful.

Is the way that Unit Tests require one to structure an application's code "only" beneficial for the unit tests or is it actually beneficial to the applications structure?

In my experience a testable design is beneficial overall, not "only" for the unit tests themselves. These benefits come on different levels:

  • Making your design testable enforces you to chunk your application into small, more or less independent parts which can only influence each other in limited and well defined ways - this is very important for the long term stability and maintainability of your program. Without this, the code tends to deteriorate into spaghetti code where any change made in any part of the codebase may cause unexpected effects in seemingly unrelated, distinct parts of the program. Which is, needless to say, is every programmer's nightmare.
  • Writing the tests themselves in TDD fashion actually exercises your APIs, classes and methods, and serves as a very effective test to detect whether your design makes sense - if writing tests against and interface feels awkward or difficult, you get valuable early feedback when it is still easy to shape the API. In other words, this defends you from publishing your APIs prematurely.
  • The pattern of development enforced by TDD helps you focus on the concrete task(s) to do, and keeps you on target, minimizing the chances of you wandering off solving other problems than the one you are supposed to, adding unnecessary extra features and complexity, etc.
  • The fast feedback of unit tests allows you to be bold in refactoring the code, enabling you to constantly adapt and evolve the design over the lifetime of the code, thus effectively preventing code entropy.

I remember a rule of thumb that said don't generalize until you need to / until there's a second place that uses the code. With Unit Tests, there's always a second place that uses the code -- namely the unit test. So is this reason enough to generalize?

If you can prove that your software does precisely what it's supposed to do - and prove it in a fast, repeatable, cheap and deterministic enough way to satisfy your customers - without the "extra" generalization or seams forced by unit tests, go for it (and let us know how you do it, because I am sure a lot of people on this forum would be as interested as me :-)

Btw I assume by "generalization" you mean things like introducing an interface (abstract class) and polymorphism instead of a single concrete class - if not, please clarify.

Related Topic