I’m asking this question regarding problems I have experienced during TDD projects.
I have noticed the following challenges when creating unit tests.
- Generating and maintaining mock data
It’s hard and unrealistic to maintain large mock data. It’s is even harder when database structure undergoes changes.
- Testing GUI
Even with MVVM and ability to test GUI, it takes a lot of code to reproduce the GUI scenario.
- Testing the business
I have experience that TDD works well if you limit it to simple business logic. However complex business logic is hard to test since the number of combinations of tests (test space) is very large.
- Contradiction in requirements
In reality it’s hard to capture all requirements under analysis and design. Many times one note requirements lead to contradiction because the project is complex. The contradiction is found late under implementation phase. TDD requires that requirements are 100% correct. In such cases one could expect that conflicting requirements would be captured during creating of tests. But the problem is that this isn’t the case in complex scenarios.
I have read this question: Why does TDD work?
Does TDD really work for complex enterprise projects, or is it practically limit to project type?
Best Answer
False.
Unit testing doesn't require "large" mock data. It requires enough mock data to test the scenarios and nothing more.
Also, the truly lazy programmers ask the subject matter experts to create simple spreadsheets of the various test cases. Just a simple spreadsheet.
Then the lazy programmer writes a simple script to transform the spreadsheet rows into unit test cases. It's pretty simple, really.
When the product evolves, the spreadsheets of test cases are updated and new unit tests generated. Do it all the time. It really works.
What? "Reproduce"?
The point of TDD is to Design things for Testability (Test Drive Development). If the GUI is that complex, then it has to be redesigned to be simpler and more testable. Simpler also means faster, more maintainable and more flexible. But mostly simpler will mean more testable.
That can be true.
However, asking the subject matter experts to provide the core test cases in a simple form (like a spreadsheet) really helps.
The spreadsheets can become rather large. But that's okay, since I used a simple Python script to turn the spreadsheets into test cases.
And. I did have to write some test cases manually because the spreadsheets were incomplete.
However. When the users reported "bugs", I simply asked which test case in the spreadsheet was wrong.
At that moment, the subject matter experts would either correct the spreadsheet or they would add examples to explain what was supposed to happen. The bug reports can -- in many cases -- be clearly defined as a test case problem. Indeed, from my experience, defining the bug as a broken test case makes the discussion much, much simpler.
Rather than listen to experts try to explain a super-complex business process, the experts have to produce concrete examples of the process.
Not using TDD absolutely mandates that the requirements be 100% correct. Some claim that TDD can tolerate incomplete and changing requirements, where a non-TDD approach can't work with incomplete requirements.
If you don't use TDD, the contradiction is found late under implementation phase.
If you use TDD the contradiction is found earlier when the code passes some tests and fails other tests. Indeed, TDD gives you proof of a contradiction earlier in the process, long before implementation (and arguments during user acceptance testing).
You have code which passes some tests and fails others. You look at only those tests and you find the contradiction. It works out really, really well in practice because now the users have to argue about the contradiction and produce consistent, concrete examples of the desired behavior.