Agile – How to cope with a team who tends to underestimate time needed to complete tasks

It looks like you took some fancy items from agile development, put them to waterfall process and now you call it agile.

The product is developed for a customer who will re-sell it while paying us royalty.

This is OK.

The team does not get to talk directly to the end user. Only to the reseller.

This is OK. Product owner talk to reseller and collects requirements.

A product requirements document was created before starting development.

This is not OK. I haven't seen the project where definite requirement set can be defined upfront. Change your product requirements document to product vision (short) with some initial set of requirements which are subject to change.

The requirements are rigid and do not change.

This is not OK and you will see in the future that it is also not true.

A delivery schedule was agreed on with milestones such as "alpha", "beta" etc. and features/times attached to those milestones.

This is not OK. The real schedule will be visible from the team progress. You can make general milestones but assigning exact set of features which will be implemented in these milestones is not agile. This can change during development.

All developers on the Scrum team report to the product owner, a software manager.

This is not OK. I would not say that developers report to product owner. Scrum process keeps visibility of the process but developers do not report anything except regular meetings. It is responsibility of product owner to be in contact with a team and as active participant see the progress himself.

Testers on the team report to a QA manager.

This is not OK. Testers should be part of development team because user story is not done until it is tested (there should be automated test to validate acceptance criteria). There can be separate QA but it is additional level of complex testing and it is usually done on customer side (but doesn't have to be) to validate that SW does what customer expects and the feedback is collected as new backlog items or bugs to existing completed backlog items.

Separating complete QA outside of development team leads to breaking the whole purpose of definition of done. Some QA must be part of the team and that part is not related to any QA manager - that part is doing commitment with development team.

The product owner has directed the team towards certain high risk technical tasks. The output of those tasks is not usable by the end user but rather some technology/code that will eventually be used in the product.

This happens in every project but it should be part of some product backlog item targeting end user. It can be included directly in backlog item implemented in current iteration or it can be included as a spike (proof-of-concept) to clarify complexity of some backlog item which should be implemented in the future.

The product owner has created a backlog based on the requirements.

This is a must.

The product owner is unable to answer some questions regarding the product. He refers to others or to the documented requirements.

This is not OK. It is job of the product owner to know answers. He has a responsibility and he must do decisions. If he doesn't know answer he must find it asap.

The team goes through the motions of Scrum. Daily Scrum, Sprint Planning, Retrospective etc. There is a ScrumMaster.

This is OK but it doesn't mean that team is doing Scrum.

Every sprint the product owner and management decide what backlog items the team works on.

This is definitely not OK. The product owner and management can make priorities but commitment (selection of most prioritized items) is teams responsibility.

There is a burndown chart. Scrum board with stories and tasks. The estimates on those come from the team.

This is OK.

The team sits in an open floor "bull pen" shared with other teams, all visible and audible. There is cross-team noise and there is foot traffic around the team area.

It is Scrum master's responsibility to make end of this if team feels like it reduces their productivity.

The team may be required to attend various meetings not directly related to the goals of the sprint.

It is OK, the time wasted on these meetings will result in smaller commitment (team will do less real work). It is up to Scrum master / management to reduce these meetings to increase team's velocity.

There are pressures to select certain technical solutions. Some tools and processes are mandated.

This is partially OK. There can be non-functional requirements for tools and architecture and there can be defined processes but still final implementation is up to the team.

We took a very similar approach by designating a "Support Programmer" that rotated through the more junior (or newest) developers on the team. They were encouraged to try really hard to figure out the problem before asking the other developers even if it would be faster to kick it over to the developer who knew the code best. This way they were forced to learn the codebase and you avoided knocking people out of "the zone" and destroying productivity. Also, you have to build in some mechanism to keep people from doing an end-around to the non-support programmers to make this work.

A key point is that the team needs to understand that what may be the fastest way to close support issues is not always the most efficient use of the team's time. Make sure the entire team knows that the goal of this structure is to minimize interruptions to people who are in that productive mode at all costs.

That said, the support programmer was not 100% dedicated to support calls. They did work on cases in the sprint, but were supposed to work on the lowest priority items so that if they got hung up on a lot of support issues the fact that they didn't finish their cases wouldn't be such a major concern.