When doing unit tests the "proper" way, i.e. stubbing every public
call and return preset values or mocks, I feel like I'm not actually
testing anything. I'm literally looking at my code and creating
examples based on the flow of logic through my public methods.
This sounds like the method you are testing needs several other class instances (which you have to mock), and calls several methods on its own.
This type of code is indeed difficult to unit-test, for the reasons you outline.
What I have found helpful is to split up such classes into:
- Classes with the actual "business logic". These use few or no calls to other classes and are easy to test (value(s) in - value out).
- Classes that interface with external systems (files, database, etc.). These wrap the external system and provide a convenient interface for your needs.
- Classes that "tie everything together"
Then the classes from 1. are easy to unit-test, because they just accept values and return a result. In more complex cases, these classes may need to perform calls on their own, but they will only call classes from 2. (and not directly call e.g. a database function), and the classes from 2. are easy to mock (because they only expose the parts of the wrapped system that you need).
The classes from 2. and 3. cannot usually be meaningfully unit-tested (because they don't do anything useful on their own, they are just "glue" code). OTOH, these classes tend to be relatively simple (and few), so they should be adequately covered by integration tests.
An example
One class
Say you have a class which retrieves a price from a database, applies some discounts and then updates the database.
If you have this all in one class, you'll need to call DB functions, which are hard to mock. In pseudocode:
1 select price from database
2 perform price calculation, possibly fetching parameters from database
3 update price in database
All three steps will need DB access, so a lot of (complex) mocking, which is likely to break if the code or the DB structure changes.
Split up
You split into three classes: PriceCalculation, PriceRepository, App.
PriceCalculation only does the actual calculation, and gets provided the values it needs. App ties everything together:
App:
fetch price data from PriceRepository
call PriceCalculation with input values
call PriceRepository to update prices
That way:
- PriceCalculation encapsulates the "business logic". It's easy to test because it does not call anything on its own.
- PriceRepository can be pseudo-unit-tested by setting up a mock database and testing the read and update calls. It has little logic, hence few codepaths, so you do not need too many of these tests.
- App cannot be meaningfully unit-tested, because it is glue-code. However, it too is very simple, so integration testing should be enough. If later App gets too complex, you break out more "business-logic" classes.
Finally, it may turn out PriceCalculation must do its own database calls. For example because only PriceCalculation knows which data its needs, so it cannot be fetched in advance by App. Then you can pass it an instance of PriceRepository (or some other repository class), custom-tailored to PriceCalculation's needs. This class will then need to be mocked, but this will be simple, because PriceRepository's interface is simple, e.g. PriceRepository.getPrice(articleNo, contractType)
. Most importantly, PriceRepository's interface isolates PriceCalculation from the database, so changes to the DB schema or data organisation are unlikely to change its interface, and hence to break the mocks.
It sort of seems like a test that says, "yes, your production code is actually working when you don't stub or mock things out."
That's exactly what it is.
Stubs and mocks isolate the infrastructure from your unit tests, which means that your units get tested, but not the infrastructure that you've stubbed out, nor the connections your units make with that infrastructure. It's the behavior of your units and the way that behavior interacts with the actual components that you've stubbed out that is being tested in an integration test.
Remember, stubs and mocks don't represent the actual behavior of a real system; they only simulate a small number of artificially-induced behaviors so that you can run your unit tests. An integration test not only provides a more realistic exercise of the system, but also a more comprehensive one.
Best Answer
Unit tests are just one level of the hierarchy of automated tests. Unit tests exist to verify that the code you (the developer) have actually written behaves the way you thought it should when you wrote it.
There are two caveats inherent in unit testing. First, coverage is not exercise. You may execute every line of code in your codebase via one or more unit tests, but if you do not assert, somewhere, that code which needed to do something actually did it, then as long as you don't get an exception, the test passes with or without the code doing this key thing.
Second, unit tests by definition exercise small, isolated pieces of your code (units), making sure each piece behaves the way the developer thinks it should. They don't test that these units play nicely with each other (that's an "integration" test), nor do they assert that the code, at any level, behaves the way the client thinks it should (that's an "acceptance" and/or an "end-to-end" test).
This second problem is your main issue in the case in point. You have 100% unit test coverage, but no integration testing, which would prove that the little pieces are put together the right way to do the larger job you expect. You also seem to have no automated acceptance testing, which approaches the entire program from the top down from the perspective of an end user. These are the levels of testing, which can be automated, that will identify your "missing" code based on failure to satisfy acceptance criteria.