What Is The Process of Creating A Bug Bar

Based on my understanding of the English language and trying to be logical about what should be the difference:

Requirements Elicitation = Requirements Gathering. This is asking what are the requirements, what if this, what if that, etc. This is about asking the questions and getting responses. How well are the answers is another matter entirely. This requires the stakeholders to answer their part of what is to be done and why.

Requirements Analysis. This is more the organizing of answers to the first part. Which solution is optimal? What are the trade-offs of various possible implementations. In this part there may be the odd question but it isn't the main point as this is about seeing which solution may be better under various constraints,e.g. which is the fastest or cheapest. This is more about how is something to be done and why does that way make more sense than another.

Another way to think of this is that the Elicitation has to come before the Analysis as otherwise you are analyzing nothing, which may not be that useful or productive.

This approach is almost always more suitable than simply interviewing the users.

Why?

Because users lie. Sometimes it's not malicious. They just can't tell a proper story.

They do this two ways.

1. They have little vision; they don't really know what a new application could possibly do. They don't describe things that include real, overall business value. Instead, they describe details of how it's implemented right now, however bad that implementation is.

   This is perhaps most common, since most users that aren't "business owners" or "executives" have been delegated a piece of a larger operation.

2. They have an overly grandiose vision; they imagine all kinds of things, some of which are impossible. They don't describe things that include real, overall business value. Instead, they fantasize about some job that exists only in their imagination.

   This is distressing because it leads to requirements that -- sometimes -- can't even be tested. Or, if they can concoct test cases, debate arises within the user organization. Or, if one organization agrees, other organizations can't provide the required data.

   Most often, however, this leads to systems that are needlessly overspecified. Often a "first sprint" provides a workable solution that does everything required but is very little like the grandiose vision.

Because users lie. Sometimes it is malicious. They refuse to tell a proper story and instead simply avoid providing concrete details. Evasive users are particular difficult to deal with, since there is no straight story. When you try to detail your understanding, you've always got several key features completely wrong -- even though you've provided verbatim words from the interview.

Best Practices

Collect actual documents (files or paper documents) that reflect the user's real workflow with real data. Nothing -- nothing -- beats actual data to provide the nuanced complexities of the daily job.

Try to discern real business value from workarounds due to bad software created by people who didn't do any job observation. It seems like 25%-50% of an application is tweaks and interfaces required to correct problems in other applications.

When observing someone solving a problem (or working around a problem) you'll have to discover the root cause. You won't discover this through job observation or interviewing. You'll discover this through a combination of observation, interviewing and code reviews.

In some cases, whole departments are created to work around software limitations. That department does not create value -- it avoids the cost of bad software. Job Observation can lead to a streamlined workaround. But it's still a workaround for a more fundamental root-cause problem.