I've seen the terms brittle and fragile used to describe poorly written unit tests. But I haven't been able to find a clear definition for what brittle or fragile actually mean in this context.
- Is it unit tests that break for more than one reason?
- Is it when a test fails because something in the code base is changed
that is supposedly unrelated to the test? - Is it tests that test functionality in an inappropriate manner, such as testing how instead of what a function does?
- Something else entirely?
Or are these terms simply a cover-all for all unit-test smells?
Best Answer
The terms "brittle" and "fragile" do not have a clear and specific definition within the context of software development. Rather, they are ordinary English words that are sometimes used in software development to describe certain kinds of code.
In the context of software development, these words mean exactly the same thing as they do everywhere else: "brittle" and "fragile" things are things that are easy to break.
That said, there is this "personal reflection essay" from Wikipedia. If you're looking for an "official" definition, this one is as good as any:
Things that make unit tests brittle:
As you can see, what makes unit tests fragile is often not the fault of the test, but the fault of the code being tested.
One way to have better tests (and by extension, better code), is to practice Test-First Development. Writing your tests first forces you to think about the surface area of your code's API, and strongly encourages you to write that surface area in a way that makes it simpler, more robust, more deterministic and easier to test.
Further Reading
The Brittleness of Unit Tests
Some Good Advice for Newcomers to Unit Testing