TL;DR
Writing good, useful tests is hard, and has a high cost in C++. Can you experienced developers share your rationale on what and when to test?
Long story
I used to do test-driven development, my whole team in fact, but it didn't work well for us. We have many tests, but they never seem to cover the cases where we have actual bugs and regressions – which usually occur when units are interacting, not from their isolated behaviour.
This is often so hard to test on the unit level that we stopped doing TDD (except for components where it really speeds up development), and instead invested more time increasing the integration test coverage. While the small unit tests never caught any real bugs and were basically just maintenance overhead, the integration tests have really been worth the effort.
Now I've inherited a new project, and am wondering how to go about testing it. It's a native C++/OpenGL application, so integration tests are not really an option. But unit testing in C++ is a bit harder than in Java (you have to explicitely make stuff virtual), and the program isn't heavily object oriented, so I can't mock/stub some stuff away.
I don't want to rip apart and OO-ize the whole thing just to write some tests for the sake of writing tests. So I'm asking you: What is it I should write tests for? e.g.:
- Functions/Classes that I expect to change frequently?
- Functions/Classes that are more difficult to test manually?
- Functions/Classes that are easy to test already?
I began to investigate some respectful C++ code bases to see how they go about testing. Right now I'm looking into the Chromium source code, but I'm finding it hard to extract their testing rationale from the code. If anyone has a good example or post on how popular C++ users (guys from the committee, book authors, Google, Facebook, Microsoft, …) approach this, that'd be extra helpful.
Update
I have searched my way around this site and the web since writing this. Found some good stuff:
- When is it appropriate to not unit test?
- https://stackoverflow.com/questions/109432/what-not-to-test-when-it-comes-to-unit-testing
- http://junit.sourceforge.net/doc/faq/faq.htm#best
Sadly, all of these are rather Java/C# centric. Writing lots of tests in Java/C# is not a big problem, so the benefit usually outweights the costs.
But as I wrote above, it's more difficult in C++. Especially if your code base is not-so-OO, you have to severely mess things up to get a good unit test coverage. For instance: The application I inherited has a Graphics name space that is a thin layer above OpenGL. In order to test any of the entities – which all use its functions directly – I'd have to turn this into an interface and a class and inject it in all the entities. That's just one example.
So when answering this question, please keep in mind that I have to make a rather big investment for writing tests.
Best Answer
Well, Unit Testing is only one part. Integration tests help you with the problem of your team. Integration Tests can be written for all kinds of applications, also for native and OpenGL applications. You should check out "Growing Object Oriented Software Guided by Tests" by Steve Freemann and Nat Pryce (e.g. http://www.amazon.com/Growing-Object-Oriented-Software-Guided-Signature/dp/0321503627). It leads you step by step through the development of an application with GUI and network communication.
Testing Software that was not test driven is another story. Check Michael Feathers "Working Effectively with Legacy Code" (http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052).