It’s hard and unrealistic to maintain large mock data. It’s is even harder when database structure undergoes changes.
False.
Unit testing doesn't require "large" mock data. It requires enough mock data to test the scenarios and nothing more.
Also, the truly lazy programmers ask the subject matter experts to create simple spreadsheets of the various test cases. Just a simple spreadsheet.
Then the lazy programmer writes a simple script to transform the spreadsheet rows into unit test cases. It's pretty simple, really.
When the product evolves, the spreadsheets of test cases are updated and new unit tests generated. Do it all the time. It really works.
Even with MVVM and ability to test GUI, it’s takes a lot of code to reproduce the GUI scenario.
What? "Reproduce"?
The point of TDD is to Design things for Testability (Test Drive Development). If the GUI is that complex, then it has to be redesigned to be simpler and more testable. Simpler also means faster, more maintainable and more flexible. But mostly simpler will mean more testable.
I have experience that TDD works well if you limit it to simple business logic. However complex business logic is hard to test since the number of combination of test (test space) is very large.
That can be true.
However, asking the subject matter experts to provide the core test cases in a simple form (like a spreadsheet) really helps.
The spreadsheets can become rather large. But that's okay, since I used a simple Python script to turn the spreadsheets into test cases.
And. I did have to write some test cases manually because the spreadsheets were incomplete.
However. When the users reported "bugs", I simply asked which test case in the spreadsheet was wrong.
At that moment, the subject matter experts would either correct the spreadsheet or they would add examples to explain what was supposed to happen. The bug reports can -- in many cases -- be clearly defined as a test case problem. Indeed, from my experience, defining the bug as a broken test case makes the discussion much, much simpler.
Rather than listen to experts try to explain a super-complex business process, the experts have to produce concrete examples of the process.
TDD requires that requirements are 100% correct. In such cases one could expect that conflicting requirements would be captured during creating of tests. But the problem is that this isn’t the case in complex scenario.
Not using TDD absolutely mandates that the requirements be 100% correct. Some claim that TDD can tolerate incomplete and changing requirements, where a non-TDD approach can't work with incomplete requirements.
If you don't use TDD, the contradiction is found late under implementation phase.
If you use TDD the contradiction is found earlier when the code passes some tests and fails other tests. Indeed, TDD gives you proof of a contradiction earlier in the process, long before implementation (and arguments during user acceptance testing).
You have code which passes some tests and fails others. You look at only those tests and you find the contradiction. It works out really, really well in practice because now the users have to argue about the contradiction and produce consistent, concrete examples of the desired behavior.
There are already lots of good answers to this question, and I've commented and upvoted several of them. Still, I'd like to add some thoughts.
Flexibility isn't for novices
The OP clearly states that he's not experienced with TDD, and I think a good answer must take that into account. In the terminology of the Dreyfus model of skill acquisition, he's probably a Novice. There's nothing wrong with being a novice - we are all novices when we start learning something new. However, what the Dreyfus model explains is that novices are characterized by
- rigid adherence to taught rules or plans
- no exercise of discretionary judgement
That's not a description of a personality deficiency, so there's no reason to be ashamed of that - it's a stage we all need to go through in order to learn something new.
This is also true for TDD.
While I agree with many of the other answers here that TDD doesn't have to be dogmatic, and that it can sometimes be more beneficial to work in an alternative way, that doesn't help anyone just starting out. How can you exercise discretionary judgement when you have no experience?
If a novice accepts the advice that sometimes it's OK not to do TDD, how can he or she determine when it's OK to skip doing TDD?
With no experience or guidance, the only thing a novice can do is to skip out of TDD every time it becomes too difficult. That's human nature, but not a good way to learn.
Listen to the tests
Skipping out of TDD any time it becomes hard is to miss out of one of the most important benefits of TDD. Tests provide early feedback about the API of the SUT. If the test is hard to write, it's an important sign that the SUT is hard to use.
This is the reason why one of the most important messages of GOOS is: listen to your tests!
In the case of this question, my first reaction when seeing the proposed API of the Yahtzee game, and the discussion about combinatorics that can be found on this page, was that this is important feedback about the API.
Does the API have to represent dice rolls as an ordered sequence of integers? To me, that smell of Primitive Obsession. That's why I was happy to see the answer from tallseth suggesting the introduction of a Roll
class. I think that's an excellent suggestion.
However, I think that some of the comments to that answer get it wrong. What TDD then suggests is that once you get the idea that a Roll
class would be a good idea, you suspend work on the original SUT and start working on TDD'ing the Roll
class.
While I agree that TDD is more aimed at the 'happy path' than it's aimed at comprehensive testing, it still helps to break the system down into manageable units. A Roll
class sounds like something you could TDD to completion much more easily.
Then, once the Roll
class is sufficiently evolved, would you go back to the original SUT and flesh it out in terms of Roll
inputs.
The suggestion of a Test Helper doesn't necessarily imply randomness - it's just a way to make the test more readable.
Another way to approach and model input in terms of Roll
instances would be to introduce a Test Data Builder.
Red/Green/Refactor is a three-stage process
While I agree with the general sentiment that (if you are sufficiently experienced in TDD), you don't need to stick to TDD rigorously, I think it's pretty poor advice in the case of a Yahtzee exercise. Although I don't know the details of the Yahtzee rules, I see no convincing argument here that you can't stick rigorously with the Red/Green/Refactor process and still arrive at a proper result.
What most people here seem to forget is the third stage of the Red/Green/Refactor process. First you write the test. Then you write the simplest implementation that passes all tests. Then you refactor.
It's here, in this third state, that you can bring all your professional skills to bear. This is where you are allowed to reflect on the code.
However, I think it's a cop-out to state that you should only "Write the simplest thing possible that isn't completely braindead and obviously incorrect that works". If you (think you) know enough about the implementation on beforehand, then everything short of the complete solution is going to be obviously incorrect. As far as advice goes, then, this is pretty useless to a novice.
What really should happen is that if you can make all tests pass with an obviously incorrect implementation, that's feedback that you should write another test.
It's surprising how often doing that leads you towards an entirely different implementation than the one you had in mind first. Sometimes, the alternative that grows like that may turn out to be better than your original plan.
Rigour is a learning tool
It makes a lot of sense to stick with rigorous processes like Red/Green/Refactor as long as one is learning. It forces the learner to gain experience with TDD not just when it's easy, but also when it's hard.
Only when you have mastered all the hard parts are you in a position to make an informed decision on when to deviate from the 'true' path. That's when you start forming your own path.
Best Answer
What a repository does is translate from your domain onto your DAL framework, such as NHibernate or Doctrine, or your SQL-executing classes. This means that your repository will call methods on said framework to perform its duties: your repository constructs the queries needed to fetch the data. If you're not using an ORM-framework (I hope your are...), the repository would be the place where raw SQL-statements are built.
The most basic of these methods is the save: in most cases this will simply pass the object from the repository onto the unit of work (or the session).
But let's look at another example, for example fetching a car by its ID. It might look like
Still not too complex, but you can imagine with multiple conditions (get me all the cars made after 2010 for all brands in the 'Volkswagen' group) this gets tricky. So in true TDD fashion you need to test this. There are several ways to do this.
Option 1: Mock the calls made to the ORM framework
Sure, you can mock the Session-object and simply assert that the right calls are made. While this tests the repository, it is not really test-driven because you are just testing that the repository internally looks the way you want it to. The test basically says 'the code should look like this'. Still, it is a valid approach but it feels like this kind of test has very little value.
Option 2: (Re)build the database from the tests
Some DAL-frameworks give you the ability to build the complete structure of the database based on the mapping files you create to map the domain onto the tables. For these frameworks the way to test repositories is often to create the database with an in-memory database in the first step of the test and add objects using the DAL-framework to the in-memory database. After this, you can use the repository on the in-memory database to test if the methods work. These tests are slower, but very valid and drive your tests. It does require some cooperation from your DAL-framework.
Option 3: Test on an actual database
Another approach is to test on an actual database and isolate the unittest. You can do this in several ways: surround your tests with a transaction, clean up manually (would not recommend as very hard to maintain), completely rebuild the database after each step... Depending on the application you are building this may or may not be feasible. In my applications I can completely build a local development database from source control and my unittests on repositories use transactions to fully isolate the tests from each other (open transaction, insert data, test repository, rollback transaction). Every build first sets up the local development database and then performs transaction-isolated unittests for the repositories on that local development database. It's a little slower then a pure Unittest but the tests are extremely valuable and catch a lot of issues.
Don't test the DAL
If you are using a DAL framework such as NHibernate, avoid the need to test that framework. You could test your mapping files by saving, retrieving and then comparing a domain object to make sure everything is okay (be sure to disable any sort of caching) but it's not as required as a lot of other tests you should be writing. I tend to do this mostly for collections on parents with conditions on the children.
When testing the return of your repositories you could simply check to see if some identifying property on your domain object matches. This can be an id but in tests it's often more beneficial to check a human readable property. In the 'get me all the cars made after 2010....' this could simply check that five cars are returned and the license plates are 'insert list here'. Added benefit is that it forces you to think about sorting AND your test automatically forces the sorting. You'd be surprised how many applications either sort multiple times (return sorted from the database, sort before creating a view object and then sort the view object, all on the same property just in case) or implicitly assume the repository sorts and accidentally remove that somehwere along the way, breaking the UI.
'Unit test' is just a name
In my opinion, unit tests should mostly not hit the database. You build an application so that every piece of code that needs data from a source does this with a repository, and that repository is injected as a dependency. This allows for easy mocking and all the TDD-goodness you want. But in the end you want to make sure that your repositories perform their duties and if the easiest way to do that is hit a database, well, so be it. I've long let go of the notion that 'unit tests should not touch the database' and learned that there are very real reasons to do this. But only if you can do this automatically and repeatedly. And weather we call such a test a 'unit test' or an 'integration test' is moot.