I feel the accepted answer was one of the weakest (Disadvantages of Test Driven Development?), and the most up-modded answer smells of someone who might be writing over specified tests.
Big time investment: for the simple
case you lose about 20% of the actual
implementation, but for complicated
cases you lose much more.
TDD is an investment. I've found that once I was fully into TDD, the time I lost is very very little, and what time I did lose was more than made up when it came to maintence time.
For complex cases your test cases are
harder to calculate, I'd suggest in
cases like that to try and use
automatic reference code that will run
in parallel in the debug version /
test run, instead of the unit test of
simplest cases.
If your test are becoming very complex, it might be time to review your design. TDD should lead you down the path smaller, less complex units of code working together
Sometimes you the design is not clear at the start and evolves as you go along - this will force you to redo your test which will generate a big time lose. I would suggest postponing unit tests in this case until you have some grasp of the design in mind.
This is the worst point of them all! TDD should really be "Test Driven Design". TDD is about design, not testing. To fully realise the value of benefits of TDD, you have toy drive your design from your tests. So you should be redoing your production code to make your tests pass, not the other way round as this point suggests
Now the currently most upmodded: Disadvantages of Test Driven Development?
When you get to the point where you have a large number of tests, changing the system might require re-writing some or all of your tests, depending on which ones got invalidated by the changes. This could turn a relatively quick modification into a very time-consuming one.
Like the accepted answers first point, this seems like over specification in the tests and a general lack of understanding of the TDD process. When making changes, start from your test. Change the test for what the new code should do, and make the change. If that change breaks other tests, then your tests are doing what their supposed to do, failing. Unit Tests, for me, are designed to fail, hence why the RED stage is first, and should never be missed.
Specs and ScalaTest are both good tools with happy users, but they differ in several ways. You will probably want to pick one as your main testing tool in Scala, but need not give up the other because you can use pieces of both. If you like ScalaTest's FeatureSpec
syntax and specs' Mockito syntax, for example, you can put both jar files in your classpath and use both at the same time. Here I'll try and capture the main design philosophy differences I've noticed between specs and ScalaTest.
Probably the main philosophical difference between the tools is that specs is designed for Behavior-Driven Development (BDD), whereas ScalaTest is more general. ScalaTest provides traits that you can mix together to get the behavior you prefer in your test classes, including BDD, and you can also easily define your own behavior if you want something different.
ScalaTest supports BDD through its Spec
, FeatureSpec
, WordSpec
, FlatSpec
, and GivenWhenThen
traits, and also has traits that you can mix in to get a nice matcher syntax. If you like "should", you mix in ShouldMatchers. If you like "must", you mix in MustMatchers
. But if you like BDD but don't like matcher syntax, you can just use one of ScalaTest's Spec traits without mixing in a matchers trait. Specs has a Specification class that you extend, and you must use the word "must" in your matcher expressions. A big philosophical difference that is evident here is that ScalaTest gives you a lot more choices. To make this space of choice easier to navigate, I provide a decision tree here:
http://www.scalatest.org/quick_start
The matcher syntax is also different between ScalaTest and specs. In ScalaTest I tried to see how far I could go with operator notation, and ended up with matcher expressions that read very much like English sentences, with spaces between the words. Specs matcher syntax runs words together more with camel case.
Specs has more matchers than ScalaTest, and that I think reflects a difference in design attitude. I actually cut probably 2/3 of the matcher syntax I built and considered for release. I will add more matchers in future releases, but wanted to be sure I knew users actually wanted something before I added it. However ScalaTest's matchers includes a dynamic property matcher syntax takes up some of that slack. For example in Specs you can write on a java.io.File
:
file must beDirectory
This will invoke the isDirectory
and make sure it is true. ScalaTest does not have any special matchers for java.io.Files
currently, but in ScalaTest, you could just use a dynamic check like this:
file must be a ('directory)
Anytime you pass a symbol in after be
, it will use reflection to look for (in this case) a method or field named directory
or a method named isDirectory
. There's also a way to make this static, by defining a BePropertyMatcher
(which requires only 2 or 3 lines of code usually). So basically in ScalaTest I try to provide more functionality with less API.
Another general design attitude difference between specs and ScalaTest
involves implicit conversions. By default you get only one implicit conversion when you use ScalaTest, which is the one that puts the ===
operator on everything. (If you need to, you can "turn off" this implicit conversion with one line of code. The only reason you would need to do that is if you were trying to test something that has its own ===
operator, and you get a conflict.) ScalaTest defines many other implicit conversions, but to use them you need to explicitly "invite" them into your code by mixing in a trait or doing an import. When you extend class Specification
in specs I think you pretty much get dozens of implicit conversions by default. I'm not sure how much that will matter in practice, but I figure people will want to test code that uses their own implicits, and sometimes there may be a conflict between the test framework's implicits and those of the production code. When that happens I think it may be easier to work around the problem in ScalaTest than specs.
Another difference in design attitude that I've noticed is comfort with operators. One goal I had was that any programmer looking at someone else's test code that uses ScalaTest would be able to guess what the meaning was without looking anything up in the ScalaTest documentation. I wanted ScalaTest client code to be drop dead obvious. One way that goal manifested itself is that ScalaTest is very conservative about operators. I only define five operators in ScalaTest:
===
, which means equals
>
, which means greater than
<
, less than
>=
, greater than or equal
<=
, less than or equal.
That's it. So these things pretty much look like what mean. If you see in someone else's code:
result should be <= 7
My hope is that you won't need to run to the API documentation to guess what that <=
means. By contrast, specs is much freer with operators. Nothing wrong with that, but it is a difference. Operators can make code more concise, but the tradeoff is you may have to run to the documentation when you find things like ->-
, >>
, |
, |>
, !
, or ^^^
(which all have special meanings in Specs) in your colleague's test code.
One other philosophical difference is that I do try and make it just slightly easier in ScalaTest to use a functional style when you need to share a fixture, whereas Specs by default continues the tradition of the setUp
and tearDown
approach popularized by JUnit, in which you reassign vars before each test. However if you want to test that way, it is also very easy in ScalaTest. You just need to mix in the BeforeAndAfter
trait.
For more insight into ScalaTest, you can watch the "Get Higher with ScalaTest" presentation I gave at the 2009 Devoxx conference here:
http://parleys.com/play/514892260364bc17fc56bde3/chapter0/about
Best Answer
When I started to look at BDD I investigated all the frameworks out there (for .net) and ended up using none of them. Main reason is I feel like the community hasn't settled on a syntax and best practises yet so instead I continued to use NUnit with a base class based on a blog post by Ben Scheirman. This worked out really well because BDD is not about the tooling but making the tests clean and understandable which is totally possible with normal tools like nunit.
Compared to my old unit tests the new style is much more readable and puts a lot more focus on naming and behavior. We're not that far from printing out the method names and have a discussion with the business people about the system.
Some additional reading by Scott Bellware: Behavior-Driven Development
Examle of a test: