bddui
I understand it is better to automate BDD specifications below the user interface whenever possible.
For example and quoting from Gojko Adzic's book: Specification by Example:
Don't check business logic through the user interface
My question is then: What are the specific use cases when using the UI layer does make sense in the context of BDD?
If you can derive the scenarios from the description, you're done.
An anti-pattern that I often see in BDD is people feeling the need to talk through, and write down, every scenario in detail.
Some scenarios are so well understood that it's enough to derive them from a brief description. For instance, if I say, "I'd like the login feature this week," you know what that should look like. You know that there are scenarios for the right password, the wrong password, the wrong username. We don't really need to talk through those or capture them in detail.
Similarly, I might say, "Here's the form for user registration. We need to be able to create new users, let them edit their details, and delete themselves, except that deletion shouldn't actually delete, it should just mark them as deleted so they can recover their accounts if they want to."
And you can ask, "Is account recovery part of this feature?"
"They can be two features if you want."
"Okay, so we have scenarios for create, read, update, delete; that should be easy enough. Let's talk about account recovery; that sounds more interesting."
In general, if the description of behavior is enough for the dev team to derive the scenarios, you don't need to talk through them. You can do so if there's any doubt, but you may just want to capture which scenarios you need to remember, if you capture any at all.
If you've never done it before or you're uncertain, talk through the scenarios.
Focus on the areas which are unusual, particularly if there are features you've never done before. These are fantastic places to have conversations and write down any surprising examples which come up. I usually have two questions I ask, based on the BDD template:
Given a context
When an event happens
Then an outcome should occur.
If everyone at the table is looking bored, the feature you're talking through is probably well understood. It's often enough to say, "It should work like X, but with Y instead." This is what Dan North calls the Ginger Cake pattern; it's like the recipe for chocolate cake, but with ginger instead of chocolate.
Even if the business stakeholder is able to derive the scenarios himself, it's really important for the dev team to be able to talk to him, pick up and internalise his language. That language then gets carried into the code, enabling them to have better conversations in the future, and helping newcomers to the project understand what's going on. If the devs don't get to speak the language, they won't use it.
If the business stakeholder or analyst really doesn't want to spend the time capturing things in the session, I'd rather the developers wrote the scenarios down in collaboration with the testers, then asked him to review it. This is more likely to uncover misunderstandings than the other way round.
Sometimes BDD doesn't work.
Another possibility is that you find a scenario the business stakeholder is uncertain about. "Oh, I hadn't thought of that! I'm not sure." Rather than trying to nail the business down and punish the business with certainty, it may be worth abandoning BDD at this point and trying something simple out to get some feedback and give the business something over which they can iterate. Keep it easy to change, and write the scenarios once there's a better understanding of what's going on.
BDD done well can really help to uncover places of uncertainty. Since every project worth doing has some aspect of it that's new and has never been done before, there is some uncertainty in there, somewhere. If you focus on using the scenarios to help deliberately discover ignorance, you'll learn faster, and learning is usually a large part of the time spent on a project.
Additionally I've found that the more dev teams collaborate in this way, the more the business are prepared to trust them with uncertainty, and the more innovation starts to occur. Innovative companies, by their very nature, have plenty of uncertainty in their projects.
I wrote a blog post a while back on Cynefin, which I find really helps me understand where the conversations will be most effective. If you read it and understand the four domains, here are the rules I use:
Simple and complicated stuff (known) is often well-understood and you don't need to talk through the scenarios in detail.
Highly complex stuff (unknown) is not understood at all. You may discover this by talking through the scenarios. The lack of certainty means that BDD won't work here, so iterate over something easy to change and get fast feedback instead. Any practice which retains your options, like A-B testing, is also great in this space.
BDD works brilliantly in the space in between (knowable) as a mechanism for passing on knowledge, and to uncover the other two spaces. It's not a hammer, and not everything is a nail. In fact, if you can focus the time spent having conversations on anything, it's not about the examples you can find; it's about finding the examples you can't.
It's perfectly possible to do this. I've written a blog with a few guidelines, but these additional ones might help you too:
Think about the capabilities of the system. For instance, an accounting system might have the ability to read bank feeds, raise invoices, email those invoices, etc.
Group the scenarios in terms of the capabilities. Look at what kind of contexts (the givens) produce what kinds of outcomes (the thens). Have some conversations with the business people about these, and pick up their language as far as possible. The capabilities themselves will drive what you write for the events (the whens).
For instance, you might find a couple of scenarios for raising invoices where it says something like:
Given an organisation to bill is outside the US
When we send the invoice
Then international bank details should be included.Given an organisation to bill is within the US
When we send the invoice
Then it should include only US bank details.
These will then tie into the automation that does the more detailed steps that actually create organisations with different addresses in different countries, send the invoices, and verify that those invoices have been sent with the correct bank details. There will be far more automation steps than there will of these higher-level ones. This is commonly referred to as declarative vs. imperative language, and will help you to work out which scenarios are the most important to cover, and which are functionally equivalent.
Notice that the difference between the scenarios is called out fairly cleanly here, which it wouldn't be if there were multiple UI steps hiding that difference. The difference between the scenarios is what illustrates the behaviour.
You are likely to find bugs. It's up to you if you want to write scenarios around what the system should do. It's highly likely though that by now there are some human workarounds, so I wouldn't worry too much about this behaviour. If the application is in the wild and producing value, it's good. Make sure you get scenarios around the core capabilities written first.
Whenever you have to fix a bug, write some unit tests. This will force you to redesign your code. Regression bugs are usually caused by poor design, and adding yet more scenarios will just make the code harder to change rather than give you any more confidence in it. This is what Michael Feathers is primarily referring to here. See also the test pyramid. As your system is refactored, the number of unit tests and integration tests should be rapidly outstripping those at the UI level.
You can use the different capabilities of the system to guide you in finding the seams which Michael talks about in his book, which will help you to refactor.
Note that I don't use the word test very often. Business people will tend to talk more comfortably about the behaviour of the system when you talk about examples or scenarios in which things happen than when you talk about tests. This harks back to the origins of BDD.
Good luck!
Best Answer
There are two distinct cases that might fall under the category of "using the UI" that you mentioned above, and they have different cases when it would make sense to use them:
1: specifying something using the UI concepts, language or elements - specific use cases that would justify this would be important rendering logic (think about twitter's link display - eg if the entire link is less than 100 chars, display the link, else shorten it, or positioning elements on the screen depending on how many links an article has etc) or critical/highly risky user interface specific workflows (navigation, dynamic menus etc). "critical/highly risky" is the crucial thing here, because it's very easy to think about the UI and one of the biggest problems new teams typically have with BDD style specs is to overdo them from the UI, describing stuff that's not that important long term
2: just automating tests through the UI while the spec is still talking about underlying functionality - eg the spec talks about free delivery, but the underlying test automation uses selenium to open a browser, load the web site, purchase books, go to checkout etc (similar to what I've described in the Three Layers pattern http://gojko.net/2010/04/13/how-to-implement-ui-testing-without-shooting-yourself-in-the-foot-2/). Specific use cases that justify this are where the system is designed in such a way that the risk is spread across the user interface, and testing below the UI would not provide enough confidence to stakeholders. This is mostly a legacy system design issue, and I've used this approach when retrofitting BDD-style tests to a legacy system before an important change, or when extending a legacy system that contains a lot of risk or important logic in the user interface layer. If you have to do this on a new system, that's typically a sign of risk not being localised and might point to problems with the system design.
Outside the scope of BDD, but related to tools that people often use for BDD, writing tests (not specs in the spec-by-example sense, but just automating regression tests) through the user interface also makes sense in the cases where the risk is spread across the different layers, so excluding the UI from the test would not provide enough confidence, or when you want to automate a small number of selected face-saving tests that additionally de-risk critical workflows throughout all components. As an example, a major media company had roughly 10 such tests for their primary web site. These tests were automated using cucumber, because that is what developers could maintain easily, and they were very flaky - almost any UI change broke them, but because there was just 10 of them the maintenance costs weren't that high. Those tests were described in UI terms and they executed through the UI to ensure the widest possible coverage. They were used by developers as a quick check they had to do before saying that they are done with a piece of work, in addition to the technical unit tests and BDD-style .
As you mentioned, generally it makes a lot of sense to avoid coupling the user interface and business logic (not just for spec or testing purposes, but for maintainability as well - the UI tends to be the most volatile part of the system).