I follow TDD religiously. My projects typically have 85% or better test coverage, with meaningful test cases.
I do a lot of work with HBase, and the main client interface, HTable, is a real pain to mock. It takes me 3 or 4 times longer to write my unit tests than it does to write tests that use a live endpoint.
I know that, philosophically, tests that use mocks should take priority over tests that use a live endpoint. But mocking HTable is a serious pain, and I'm not really sure it offers much of an advantage over testing against a live HBase instance.
Everyone on my team runs a single-node HBase instance on their workstation, and we have single-node HBase instances running on our Jenkins boxes, so it's not an issue of availability. The live endpoint tests obviously take longer to run than the tests that use mocks, but we don't really care about that.
Right now, I write live endpoint tests AND mock-based tests for all my classes. I'd love to ditch the mocks, but I don't want quality to decline as a result.
What do you all think?
Best Answer
My first recommendation would be to not mock types you don't own. You mentioned HTable being a real pain to mock - maybe you should wrap it instead in an Adapter that exposes the 20% of HTable's features you need, and mock the wrapper where needed.
That being said, let's assume we're talking about types you all own. If your mock-based tests are focused on happy path scenarios where everything goes smoothly, you won't lose anything ditching them because your integration tests are probably already testing the exact same paths.
However, isolated tests become interesting when you start thinking about how your system under test should react to every little thing that could happen as defined in its collaborator's contract, regardless of the actual concrete object it's talking to. That's part of what some call basic correctness. There could be many of those little cases and many more combinations of them. This is where integration tests start getting lousy while isolated tests will remain fast and manageable.
To be more concrete, what happens if one of your HTable adapter's methods returns an empty list ? What if it returns null ? What if it throws a connection exception ? It should be defined in the Adapter's contract if any of those things could happen, and any of its consumers should be prepared to deal with these situations, hence the need for tests for them.
To sum up : you won't see any quality decline by removing your mock-based tests if they tested the exact same things as your integration tests. However, trying to imagine additional isolated tests (and contract tests) can help you think out your interfaces/contracts extensively and increase quality by tackling defects that would have been hard to think about and/or slow to test with integration tests.