I'm really struggling to write effective unit tests for a large Django project. I have reasonably good test coverage, but I've come to realize that the tests I've been writing are definitely integration/acceptance tests, not unit tests at all, and I have critical portions of my application that are not being tested effectively. I want to fix this ASAP.
Here's my problem. My schema is deeply relational, and heavily time-oriented, giving my model object high internal coupling and lots of state. Many of my model methods query based on time intervals, and I've got a lot of auto_now_add
going on in timestamped fields. So take a method that looks like this for example:
def summary(self, startTime=None, endTime=None):
# ... logic to assign a proper start and end time
# if none was provided, probably using datetime.now()
objects = self.related_model_set.manager_method.filter(...)
return sum(object.key_method(startTime, endTime) for object in objects)
How does one approach testing something like this?
Here's where I am so far. It occurs to me that the unit testing objective should be given some mocked behavior by key_method
on its arguments, is summary
correctly filtering/aggregating to produce a correct result?
Mocking datetime.now() is straightforward enough, but how can I mock out the rest of the behavior?
- I could use fixtures, but I've heard pros and cons of using fixtures for building my data (poor maintainability being a con that hits home for me).
- I could also setup my data through the ORM, but that can be limiting, because then I have to create related objects as well. And the ORM doesn't let you mess with
auto_now_add
fields manually. - Mocking the ORM is another option, but not only is it tricky to mock deeply nested ORM methods, but the logic in the ORM code gets mocked out of the test, and mocking seems to make the test really dependent on the internals and dependencies of the function-under-test.
The toughest nuts to crack seem to be the functions like this, that sit on a few layers of models and lower-level functions and are very dependent on the time, even though these functions may not be super complicated. My overall problem is that no matter how I seem to slice it, my tests are looking way more complex than the functions they are testing.
Best Answer
I'm going to go ahead and register an answer for what I've come up with so far.
My hypothesis is that for a function with deep coupling and state, the reality is that it's simply going to take a lot of lines to control for its outside context.
Here's what my test case looks like roughly, relying on the standard Mock library:
datetime
and subvert theauto_now_add
times to fit a fixed timeline of my design. I thought that the ORM didn't allow this, but it works fine.from datetime import datetime
so that I can patchdatetime.now()
in just that function (if I mock the entiredatetime
class, the ORM pitches a fit).object.key_method()
, with simple but well defined functionality that depends on the arguments. I want it to depend on the arguments, because otherwise I might not know if the logic of the function-under-test is working. In my case, it simply returns the number of seconds betweenstartTime
andendTime
. I patch it in by wrapping it in a lambda and patching directly on toobject.key_method()
using thenew_callable
kwarg ofpatch
.summary
with different arguments to check equality with expected hand-calculated results accounting for the given behavior of the mockkey_method
Needless to say, this is significantly longer and more complicated than the function itself. It depends on the DB, and doesn't really feel like a unit test. But it is also fairly decoupled from the internals of the function--just its signature and dependencies. So I think it might actually be a unit test, still.
In my app, the function is quite pivotal, and subject to refactoring to optimize its performance. So I think the trouble is worth it, complexity notwithstanding. But I'm still open to better ideas on how to approach this. All part of my long journey toward a more test-driven style of development...