Java – Unit tests vs integration tests with Spring

integration-testingjavajunitspring-mvctdd

I'm working on a Spring MVC project, and I have unit tests for all of the various components in the source tree.

For example, if I have a controller HomeController, which needs to have a LoginService injected into it, then in my unit test HomeControllerTest I simply instantiate the object as normal (outside of Spring) and inject the property:

protected void setUp() throws Exception {
    super.setUp();
    //...
    controller = new HomeController();
    controller.setLoginService( new SimpleLoginService() );
    //...
}

This works great for testing each component as an isolated unit – except now that I have a few dozen classes in the project, after writing a class and writing a successful unit test for it, I keep forgetting to update my Spring MVC context file that does the actual wiring-up in the deployed application. I find out that I forgot to update the context file when I deploy the project to Tomcat and find a bunch of NullPointers from non-wired-up beans.

So, here are my questions:

This is my first Spring project – is it normal to create unit tests for the individual beans, as I have done, and then create a second suite of tests (integration tests) to test that everything works as expected with the actual application context? Is there an established best practice for this?
In addition, how do you separate the unit tests from the integration tests? I have all of the source code in src, the unit tests in test – should there be a 2nd test folder (such as test-integration) for integration test cases?

Since this is my first Spring project, I'm curious how others usually go about doing this sort of thing – and rather than re-invent the wheel I rather ask the rest of the community.

Best Answer

I can't speak to being a best practice, but here's what I've done in the past.

Unit tests:

Create unit tests for non-trivial beans (ie, most of your Spring related beans)
Use Mocks for injected services where practical (ie, most if not all the time).
Use a standard naming convention for these tests in the project test directory. Using Test or TestCase as a prefix or suffix to the classname seems to be widely practiced.

Integration Tests:

Create an AbstractIntegrationTestCase that sets up a Spring WebApplicationContext for use in intetgration test clases.
Use a naming convention for integration tests in the test directory. I've used IntTest or IntegrationTest as a prefix or suffix for these tests.

Set up three Ant test targets:

test-all (or whatever you want to name it): Run Unit and Integration Tests
test: Run Unit tests (just because test seems to be the most common usage for unit testing
test-integration: run the integration tests.

As noted, you can use the naming conventions that make sense for your project.

As to separating unit from integration tests into a separate directory, I don't think it matters as long as the developers and their tools can find and execute them easily.

As an example, the last Java project I worked on with Spring used exactly what is described above, with integration tests and unit tests living in the same test directory. Grails projects, on the other hand, explicitly separate unit and integration test directories under a general test directory.

Related Solutions

The difference between integration and unit tests

The key difference, to me, is that integration tests reveal if a feature is working or is broken, since they stress the code in a scenario close to reality. They invoke one or more software methods or features and test if they act as expected.

On the opposite, a Unit test testing a single method relies on the (often wrong) assumption that the rest of the software is correctly working, because it explicitly mocks every dependency.

Hence, when a unit test for a method implementing some feature is green, it does not mean the feature is working.

Say you have a method somewhere like this:

public SomeResults DoSomething(someInput) {
  var someResult = [Do your job with someInput];
  Log.TrackTheFactYouDidYourJob();
  return someResults;
}

DoSomething is very important to your customer: it's a feature, the only thing that matters. That's why you usually write a Cucumber specification asserting it: you wish to verify and communicate the feature is working or not.

Feature: To be able to do something
  In order to do something
  As someone
  I want the system to do this thing

Scenario: A sample one
  Given this situation
  When I do something
  Then what I get is what I was expecting for

No doubt: if the test passes, you can assert you are delivering a working feature. This is what you can call Business Value.

If you want to write a unit test for DoSomething you should pretend (using some mocks) that the rest of the classes and methods are working (that is: that, all dependencies the method is using are correctly working) and assert your method is working.

In practice, you do something like:

public SomeResults DoSomething(someInput) {
  var someResult = [Do your job with someInput];
  FakeAlwaysWorkingLog.TrackTheFactYouDidYourJob(); // Using a mock Log
  return someResults;
}

You can do this with Dependency Injection, or some Factory Method or any Mock Framework or just extending the class under test.

Suppose there's a bug in Log.DoSomething(). Fortunately, the Gherkin spec will find it and your end-to-end tests will fail.

The feature won't work, because Log is broken, not because [Do your job with someInput] is not doing its job. And, by the way, [Do your job with someInput] is the sole responsibility for that method.

Also, suppose Log is used in 100 other features, in 100 other methods of 100 other classes.

Yep, 100 features will fail. But, fortunately, 100 end-to-end tests are failing as well and revealing the problem. And, yes: they are telling the truth.

It's very useful information: I know I have a broken product. It's also very confusing information: it tells me nothing about where the problem is. It communicates me the symptom, not the root cause.

Yet, DoSomething's unit test is green, because it's using a fake Log, built to never break. And, yes: it's clearly lying. It's communicating a broken feature is working. How can it be useful?

(If DoSomething()'s unit test fails, be sure: [Do your job with someInput] has some bugs.)

Suppose this is a system with a broken class:

A single bug will break several features, and several integration tests will fail.

A single bug will break several features, and several integration tests will fail

On the other hand, the same bug will break just one unit test.

The same bug will break just one unit test

Now, compare the two scenarios.

The same bug will break just one unit test.

All your features using the broken Log are red
All your unit tests are green, only the unit test for Log is red

Actually, unit tests for all modules using a broken feature are green because, by using mocks, they removed dependencies. In other words, they run in an ideal, completely fictional world. And this is the only way to isolate bugs and seek them. Unit testing means mocking. If you aren't mocking, you aren't unit testing.

The difference

Integration tests tell what's not working. But they are of no use in guessing where the problem could be.

Unit tests are the sole tests that tell you where exactly the bug is. To draw this information, they must run the method in a mocked environment, where all other dependencies are supposed to correctly work.

That's why I think that your sentence "Or is it just a unit test that spans 2 classes" is somehow displaced. A unit test should never span 2 classes.

This reply is basically a summary of what I wrote here: Unit tests lie, that's why I love them.

Random data in Unit Tests

There's a compromise. Your coworker is actually onto something, but I think he's doing it wrong. I'm not sure that totally random testing is very useful, but it's certainly not invalid.

A program (or unit) specification is a hypothesis that there exists some program that meets it. The program itself is then evidence of that hypothesis. What unit testing ought to be is an attempt to provide counter-evidence to refute that the program works according to the spec.

Now, you can write the unit tests by hand, but it really is a mechanical task. It can be automated. All you have to do is write the spec, and a machine can generate lots and lots of unit tests that try to break your code.

I don't know what language you're using, but see here:

Java http://functionaljava.org/

Scala (or Java) http://github.com/rickynils/scalacheck

Haskell http://www.cs.chalmers.se/~rjmh/QuickCheck/

.NET: http://blogs.msdn.com/dsyme/archive/2008/08/09/fscheck-0-2.aspx

These tools will take your well-formed spec as input and automatically generate as many unit tests as you want, with automatically generated data. They use "shrinking" strategies (which you can tweak) to find the simplest possible test case to break your code and to make sure it covers the edge cases well.

Happy testing!

Best Answer

Related Solutions

The difference between integration and unit tests

The difference

Random data in Unit Tests

Related Topic