Integration vs. unit tests
You should keep your unit tests and your integration tests completely separated. Your unit tests should test one thing and one thing only and in complete isolation of the rest of your system. A unit is loosely defined but it usually boils down to a method or a function.
It makes sense to have tests for each unit so you know their algorithms are implemented correctly and you immediately know what went wrong where, if the implementation is flawed.
Since you test in complete isolation while unit-testing you use stub and mock objects to behave like the rest of your application. This is where integration tests come in. Testing all units in isolation is great but you do need to know if the units are actually working together.
This means knowing if a model is actually stored in a database or if a warning is really issued after algorithm X fails.
Test driven development
Taking it a step back and looking at Test Driven Development (TDD) there are several things to take into account.
- You write your unit test before you actually write the code that makes it pass.
- You make the test pass, write just enough code to accomplish this.
- Now that the test passes it is time to take a step back. Is there anything to refactor with this new functionality in place? You can do this safely since everything is covered by tests.
Integration first vs Integration last
Integration tests fit into this TDD cycle in one of two ways. I know of people who like to write them beforehand. They call an integration test an end-to-end test and define an end to end test as a test that completely tests the whole path of a usecase (think of setting up an application, bootstrapping it, going to a controller, executing it, checking for the result, output, etc...). Then they start out with their first unit test, make it pass, add a second, make it pass, etc... Slowly more and more parts of the integration test pass as well until the feature is finished.
The other style is building a feature unit test by unit test and adding the integration tests deemed necessary afterwards. The big difference between these two is that in the case of integration test first you're forced to think of the design of an application. This kind of disagrees with the premise that TDD is about application design as much as about testing.
Practicalities
At my job we have all our tests in the same project. There are different groups however. The continuous integration tool runs what are marked as unit tests first. Only if those succeed are the slower (because they make real requests, use real databases, etc) integration tests executed as well.
We usually use one test file for one class by the way.
Suggested reading
- Growing object-oriented software, guided by tests This book is an extremely good example of the integration test first methodology
- The art of unit testing, with examples in dot.net On unit testing, with examples in dot.net :D Very good book on principles behind unit-testing.
- Robert C. Martin on TDD (Free articles): Do read the first two articles he linked there as well.
It is useful to have automated functional tests that test the front-end while mocking away the server, but they are not integration tests. They still test a component in isolation, nothing is integrated. They are functional tests of the component.
To qualify as integration test, the test should test that all the components of the final solution integrate, that is work together. For this it needs the complete setup. With the real server and real database loaded with realistic sample data and no mocks at all. You do want to have such test.
The integration tests (of the complete setup) are the most important. They test all the code that will be involved and you should do that before you put any trust in the application. What other tests you want is a matter of balancing the effort to create them with the benefit they bring. The main benefit of tests of individual components is that they allow easier debugging than the integration tests, so write those tests that help you in debugging. Which they are depends on complexity of setting up the various components and is therefore specific to the project.
Best Answer
Functional testing and integration testing are classifications of tests, and they are not mutually exclusive.
In this case it's both. Integration tests are by definition any test that is testing more than one component. It can be at a low level to test two classes that work together, or it can be tests that test the entire system as a whole.
Functional tests are tests that confirm a given system function. So if I have a requirement "the system shall persist users object in the database", the functional test could verify that requirement by starting the system, saving a user, stopping the system, starting it again, and verifying the user exists and has not changed.
If you're writing test cases from a requirement spec or from user stories, I would call your test cases "functional tests", but they are also classified as integration tests by nature.
From my understanding "big bang integration testing" is integration testing where all the components of the system are integrated and then tested. The goal of this test is to make sure all the components are integrating correctly and the app can start. The downside to this kind of testing is that a failure of a test case gives very little direction to where the bug is, so it's important to have lower level integration tests as well.
Your functional tests will probably also integrate all the components in the system before testing. But while big bang integration testing may look similar because the entire system is being started, they are conceptually different.