I'm considering a project to migrate part of our WCF-based SOA over to a service bus model (probably nServiceBus) and using some basic pub-sub to achieve Command-Query Separation.
I'm not new to SOA, or even to service bus models, but I confess that until recently my concept of "separation" was limited to run-of-the-mill database mirroring and replication. Still, I'm attracted to the idea because it seems to provide all the benefits of an eventually-consistent system while sidestepping many of the obvious drawbacks (most notably the lack of proper transactional support).
I've read a lot on the subject from Udi Dahan who is basically the guru on ESB architectures (at least in the Microsoft world), but one thing he says really puzzles me:
As we get larger entities with more fields on them, we also get more actors working with those same entities, and the higher the likelihood that something will touch some attribute of them at any given time, increasing the number of concurrency conflicts.
[…]
A core element of CQRS is rethinking the design of the user interface to enable us to capture our users’ intent such that making a customer preferred is a different unit of work for the user than indicating that the customer has moved or that they’ve gotten married. Using an Excel-like UI for data changes doesn’t capture intent, as we saw above.
— Udi Dahan, Clarified CQRS
From the perspective described in the quotation, it's hard to argue with that logic. But it seems to go against the grain with respect to SOAs. An SOA (and really services in general) are supposed to deal with coarse-grained messages so as to minimize network chatter – among many other benefits.
I realize that network chatter is less of an issue when you've got highly-distributed systems with good message queuing and none of the baggage of RPC, but it doesn't seem wise to dismiss the issue entirely. Udi almost seems to be saying that every attribute change (i.e. field update) ought to be its own command, which is hard to imagine in the context of one user potentially updating hundreds or thousands of combined entities and attributes as it often is with a traditional web service.
One batch update in SQL Server may take a fraction of a second given a good highly-parameterized query, table-valued parameter or bulk insert to a staging table; processing all of these updates one at a time is slow, slow, slow, and OLTP database hardware is the most expensive of all to scale up/out.
Is there some way to reconcile these competing concerns? Am I thinking about it the wrong way? Does this problem have a well-known solution in the CQS/ESB world?
If not, then how does one decide what the "right level" of granularity in a Command should be? Is there some "standard" one can use as a starting point – sort of like 3NF in databases – and only deviate when careful profiling suggests a potentially significant performance benefit?
Or is this possibly one of those things that, despite several strong opinions being expressed by various experts, is really just a matter of opinion?
Best Answer
On the topic of "every attribute change"
I think you missed the point. Mr. Udi Dahan is saying you should capture the user's intent as a command. An end-user is concerned with being able to indicate that a customer has moved. Depending on the context that command could contain a customer identification, the new address (split up into street, streetnumber, zipcode, ...), optionally a new phone number (not uncommon when you move - maybe less so with all these cellphones). That's hardly one attribute. A better question is "how do I design commands?". You design them from a behavioral perspective. Each use-case, flow, task an end-user is trying to complete, will be captured in one or more commands. What data goes with those commands comes naturally, as you start reasoning about them in more detail. The thing to watch out for is data that gets interpreted as "logic flow control" on the server side. That might be an indication that you need to split up the command. I hope you never find that standard with regard to command granularity. Good question though!