Testing – Do We Need Test Data or Rely on Unit and Manual Tests?

I think your question really has two aspects to it:

1) How does a traditional software QA/QE group fit into an Agile development environment?

In my experience, you're correct. Separating the testing team from the development team creates some issues. The "throw it over the fence" method doesn't work very well in Agile. What we've done is make the testing team just part of the Agile Team. If your quality team is more technical, maybe easing them into development is a possibility. Maybe one member of the quality team is interested in becoming the Scrum Master. Maybe some members of the quality team are looked at as specialized testing resources on the Team. I think it's important in any case that they are full-fledged Team members, though; they come to all of the standard Scrum meetings, help with estimation, contribute to the Retrospective, etc.

2) How is the testing responsibility spread throughout an Agile Team?

This has to do with your Team's Definition of Done. The testing effort for a Story should be part of the acceptance criteria. Testing (both automated and manual) should be parts of the tasks toward completion of the Story. That way, the dedicated testers can be assigned pieces of the Story that are an actual part of their completion. Of course, this means that development tasks need to be granular enough that there are testing tasks available throughout the Sprint. For testing resources, maybe the first couple of days of a Sprint is spent on maintaining automated tests, or grooming the bug database. We've decided that bugs found on a Story during the Sprint it's being developed in need to be fixed for the Story to be accepted. Bugs reported after the Story is accepted become part of the product backlog and are prioritized.

Hopefully this gives you one possible answer. As always with Agile, you'll find that what works for your Team will be unique in some respects.

To your first bullet point and last bullet point are pretty much the same thing and I'll address those last.

To your second bullet point, it is not this tests job to detect side effects of the SomeDao's store method. This test is testing SomeServiceImpl, not SomeDao, it doesn't matter what SomeDao does. Another test fixture should be testing that. The only thing that this test should be checking was whether store was called. Otherwise this is an integration test.

Third bullet point - yes, this is an unfortunate drawback of unit testing but the more complicated a system gets the more considerations we must make. But this isn't entirely a bad thing. It forces us to think about how one piece of code interacts with other pieces. It gives us a sharp glimpse of the point of interaction between them and let's us evaluate how we expect things to work.

Now the real meat. While I don't know that I'd consider expectations "repeating" code, but this is probably the biggest legitimate criticism to unit tests and mocking frameworks; the knowledge the tests often have to have about the specific method they're testing.

But that's not the end of the world. Yes if you completely change how you do data access the unit test will have to drastically change too, but that's unlikely. If your DAO has a proper interface on it and respects the open/closed principle this is an unlikely scenario. So you probably will only catch regressions or have to write more tests for new functionality rather than have to refactor for changed functionality.

So I'd say just make sure your service dependencies are well abstracted, try your best to make them closed for modification (+open for extension) and you will be in about as good a shape as you can be.