I'm looking for information on the best approach to message validation in asynchronous-messaging-based services (i.e. services that pull messages from some sort of message queue or broker, rather than providing an HTTP-based API or something similarly request-response oriented).
In an HTTP API, if the message is invalid (wrong fields, invalid JSON/XML/Whatever) you can return a 400 error. The receiving service can go on as before, and the sender has to deal with the error. (Many HTTP libraries raise an exception when a non-200 response is returned, making the source of the problem very clear.)
With async messaging processes, I can think of a few approaches:
- The receiving process deals with the error somehow.
- The receiving service just ignores messages that are not valid.
- The receiving service logs the invalid message and then moves on to the next message. If things don't seem to be working correctly, hopefully, we can locate the problem in the logs.
- The receiving service sends an error notification back on some sort of a response channel.
None of these options seem particularly good.
Option (1) is really just a "hand-wave" in the direction of some as-yet-unknown solution.
Options (2)-(3) seem to run the risk of problems in the sender's code being masked and made difficult to identify (compared to the request-response situation, where you can immediately identify a 400 response, which occurs in a "location" very closely linked to the source of the error).
Option (4) seems to be attempting to recreate a request/response system, negating any reasons for opting for an async-messaging based system. If you're going to request/response, why would you be using a message-broker based system in the first place?
I would appreciate any information on best practices, and on how the tension described above might be resolved.
Best Answer
Most* Message Oriented Middleware platforms provide a mechanism for the schematic validation (i.e. against a schema) of incoming messages as they are enqueued. This is logic that checks for things like missing/null fields, values outside of certain ranges, bad formatting, etc. But, as these checks are being done by the message broker, and not the end system - cannot check references such as customer ID exists, order exists, etc.
* "Most" of the strongly-typed message brokers, not just systems that queue dumb strings.
For example, in JMS a 'Validator' method can be assigned: https://docs.spring.io/spring/docs/4.1.1.RELEASE/spring-framework-reference/html/jms.html#jms-annotated-method-signature
I believe WCF has something similar, closest thing I could find was this article.
As for your options, I believe 4 is probably the most common. There are reasons (to do with scalability) why this response channel is preferable to a request-response driven system (I can validate your message and send you the Exception whenever, rather than tying up a socket + process/thread on each end while you wait for me to do it live, etc).