I'm going to go ahead and register an answer for what I've come up with so far.
My hypothesis is that for a function with deep coupling and state, the reality is that it's simply going to take a lot of lines to control for its outside context.
Here's what my test case looks like roughly, relying on the standard Mock library:
- I use the standard ORM to set up the sequence of events
- I create my own start
datetime and subvert the auto_now_add times to fit a fixed timeline of my design. I thought that the ORM didn't allow this, but it works fine.
- I make sure the function-under-test uses
from datetime import datetime so that I can patch datetime.now() in just that function (if I mock the entire datetime class, the ORM pitches a fit).
- I create my own replacement for
object.key_method(), with simple but well defined functionality that depends on the arguments. I want it to depend on the arguments, because otherwise I might not know if the logic of the function-under-test is working. In my case, it simply returns the number of seconds between startTime and endTime. I patch it in by wrapping it in a lambda and patching directly on to object.key_method() using the new_callable kwarg of patch.
- Finally, I run a series of asserts on various calls of
summary with different arguments to check equality with expected hand-calculated results accounting for the given behavior of the mock key_method
Needless to say, this is significantly longer and more complicated than the function itself. It depends on the DB, and doesn't really feel like a unit test. But it is also fairly decoupled from the internals of the function--just its signature and dependencies. So I think it might actually be a unit test, still.
In my app, the function is quite pivotal, and subject to refactoring to optimize its performance. So I think the trouble is worth it, complexity notwithstanding. But I'm still open to better ideas on how to approach this. All part of my long journey toward a more test-driven style of development...
It's comparing oranges and apples.
Integration tests, acceptance tests, unit tests, behaviour tests - they are all tests and they will all help you improve your code but they are also quite different.
I'm going to go over each of the different tests in my opinion and hopefully explain why you need a blend of all of them:
Integration tests:
Simply, test that different component parts of your system integrate correctly - for example - maybe you simulate a web service request and check that the result comes back. I would generally use real (ish) static data and mocked dependencies to ensure that it can be consistently verified.
Acceptance tests:
An acceptance test should directly correlate to a business use case. It can be huge ("trades are submitted correctly") or tiny ("filter successfully filters a list") - it doesn't matter; what matters is that it should be explicitly tied to a specific user requirement. I like to focus on these for test-driven development because it means we have a good reference manual of tests to user stories for dev and qa to verify.
Unit tests:
For small discrete units of functionality that may or may not make up an individual user story by itself - for example, a user story which says that we retrieve all customers when we access a specific web page can be an acceptance test (simulate hitting the web page and checking the response) but may also contain several unit tests (verify that security permissions are checked, verify that the database connection queries correctly, verify that any code limiting the number of results is executed correctly) - these are all "unit tests" that aren't a complete acceptance test.
Behaviour tests:
Define what the flow should be of an application in the case of a specific input. For example, "when connection cannot be established, verify that the system retries the connection." Again, this is unlikely to be a full acceptance test but it still allows you to verify something useful.
These are all in my opinion through much experience of writing tests; I don't like to focus on the textbook approaches - rather, focus on what gives your tests value.
Best Answer
If you look at the Model-Template-View triads, each distinct component and any intermediary component in between can be tested separately. Say you have a
Foomodel and need to add controllers for CRUD behavior in a simple and straightforward manner:Foomodel to assert that the business logic is implemented.FooFormto assertform.is_valid()is returning expected results for the inputs your tests supply directly.Foomodel andFooFormrespectively.In general, there's no extensive frontend testing happening in Django projects, especially those that utilize class-based views and unit test the views directly, you won't see any
assertContains/assertNotContainstests at all, although the LiveServerTestCase is going to change that a bit, at least when it comes to asserting behavior for project's that aren't already writing JavaScript tests in one of the testing frameworks.It's as if you have your backend hat on when your writing unit tests. If you want to add a success message to the CUD views, you'd assert that a success-level message has been added to the list of user's messages after a successful request, but you won't care whether that message is a part of the response or not. Similarly, we test that the context is populated with the correct
Fooobjects for a request, but we don't care if and how they're displayed.And this is where Lettuce comes into play, especially if you want to test your application's behavior from a user's perspective. It's appropriate to work with the content of a response object and to ignore the underlying implementations that produced the response. Here you won't care whether or not there is a success-level message associated with a user, you'll just want to make sure it's there in the response and you won't care whether the appropriate
Fooobjects are in the context, you'll want to assert that their content is present in the response.