They're not actually user stories. They're stakeholder stories. Unless the software is actually paid for direct by users, it's rare that a story is created entirely for their benefit.
I give you a couple of examples:
- keyworded articles, which allow advertisers to have more effective adverts
- CAPTCHAs, which are there to stop moderators having to deal with spam manually.
Most technical stories actually provide a business benefit, but it's rarely for the users. Phrasing them in a different way can help. I normally use Chris Matts' Feature Injection template:
In order to <achieve my goal>
As <the stakeholder who wants the goal>
I want (<some users to do>) <some stuff>.
This explicitly recognises all kinds of stakeholders, including the development team. Now you can phrase your technical stories too, calling out the business benefit:
In order to minimize the risk of deploying something broken
As the team deploying the code
We want to spend a few days on an automated deployment system.
I've written a couple of blog posts on this: They're not User Stories, and Feature Injection and handling technical stories. Hope they help.
I am not familiar with TFS (is that a software tool?) but I can give you some answers based on Schwarber.
1. Story Containers
There are only two containers of stories that you should ever be thinking about, the product backlog and the sprint backlog. Software tools for tracking scrum may allow you to keep around old sprint backlogs for historical analysis and that is ok, but you must close out an old sprint backlog (i.e. make sure everything in it is finished or moved out) before doing any work on the next sprint backlog.
Sometimes there is a tendency to create the next sprint's backlog in a software tool while still in the current sprint. Do not do this. If that future sprint backlog is there, people will try to put stories into it. This disrupts the proper planning process, raises tensions, and confuses scheduling issues.
If you fail to keep proper boundaries between your sprints then you are, in essence, running multiple, similtanious sprints. You are multi-tasking. At the very least the overhead of task switching will slow you down; research indicates that a full context switch in humans, from puzzle A to puzzle B, takes 15 minutes. My experience suggests this drag can be 50% or more of your productivity.
2. Epics
Keep your epics around. Someone asked for this feature, they will probably come back and want to know the status of the epic. That person, department, or customer will be thinking of the 'story' they submitted, now promoted to epic. They won't be thinking about the smaller stories involved and may not even recognize that story Foo is related to their request. The epic is a convenient handle for communication between development and the customer.
Since the epics don't actually get worked on themselves, just the associated stories, the epics move directly from your product backlog to your finished pile.
3. Where do the stories go?
A story should only be in one container at a time. It should start out in the product backlog, move to a sprint backlog, and then be finished. If someone comes looking for their story in the product backlog and doesn't find it, that must mean it is in process or finished.
4. Final thoughts about managing a deep product backlog
Force-ranked order becomes pretty meaningless when you have hundreds of items in your product backlog. Sure, the how you arrange items 20-70 might be fairly meaningful, but who really cares if #300 is before or after #301?
One possible solution to this is to have multiple sub-component backlogs that feed into a main product backlog. For example, you might have UI, DB, Backend, API, Infrastructure, and Technical Debt as your sub-components. Each component backlog might be delegated to a different person for management. A periodic meeting would have to decide what stories to move onto the main product backlog. In order to maintain a proper balance of stories in your product backlog, it is best to decide a priori a guideline (not rule) for the proportions of stories that get promoted to the main backlog. One API story for every two UI stories? How many stories can you take from Technical Debt in comparison to the number of Backend stories you need to take?
This system adds considerable complexity and requires lots of extra coordination. It should only be undertaken when the the product backlog grows so large that the Product Owner can't think about all of the stories at once.
Best Answer
If you need to estimate several user-stories that have some elements in common, but you don't yet know in which order these stories are to be implemented, then I would split the tasks that make up each story into three groups:
Then you estimate each task, where the common tasks should be estimated only once.
In the presentation to the customer/PO, I would give two estimates for each story: One with all the "common, needed once" tasks included and one with them excluded.
Just keep a detailed accounting of the tasks, their classification and estimation at hand if the customer wants more detailed information about the difference between the estimates. The tasks themselves are not negotiable, but knowing about them could help the customer/PO, especially if the set of common tasks is not the same for each story.