Unit Testing – How to Test the Wording of an Exception Message

exceptionsunit testing

Doing a code review, I ran into this assertion in a unit test:

assertThatThrownBy(() -> shoppingCartService.payForCart(command))
  .isInstanceOfSatisfying(PaymentException.class, 
    exception -> assertThat(exception.getMessage()).isEqualTo(
      "Cannot pay for ids [" + item.getId() +"], status is not WAITING"));

I feel that testing the literal text of an Exception is bad practice, but I'm unable to convince the author. (Note: this is not localized text; it's a String literal in the code with the item ID as a parameter.) My reasoning:

  • If we decided that, e.g. the item's ID was a critical piece of information, it might make sense to test that the message contains the ID. But in this case, it would be better to include the ID as a field of the Exception class.
  • If we had some sort of external system that was automatically reading the logs and looking for certain strings (we don't), then yes, this test would be justified
  • What's important in the message is that it clearly communicates what the problem is. But there must be hundreds of ways of constructing such a message, and no testing library can tell us if a plain text human language String is "clear" or "useful".
  • Thus, this amounts to, e.g. writing unit tests for translations – it's pointless because the best you can do boils down to duplicating your messages file in the tests.

What's the best practice here, and why?

Best Answer

I feel that testing the literal text of an Exception is bad practice

Never try to interpret the wording of an Exception message. They're just too darned easy for somebody [else] to change!

If the handlers of an specific Exception need specific data, then these should be added as custom properties on a custom Exception class (which the handlers can specifically look to catch). The handling code can then obtain the information it requires unambiguously and reliably.

OK, if the message happens to contain a textual representation of the same information, that's fine, but application code must not rely on it.

it would be better to include the ID as a field of the Exception class

Absolutely, yes.

If we had some sort of external system that was automatically reading the logs

If you were putting data into Exception messages that were being examined by a.n.other system or made available to a.n.other group of users, then you should be [very] careful about exactly what data you're sending - "logging" somebody's name and date-of-birth in the same Exception message might get you into [very] hot water, these days! (GDPR)

What's important in the message is that it clearly communicates what the problem is

Only if that message is going to wind up in front of a User.

To me, Structured Exception Handling is all about the Handling - doing something "useful" with each Exception. In this context, having the application throw it up in front of the User and "give up" is the very loosest definition of "useful"; a last resort.

Ideally, you want [code] to handle the Exception and make it "disappear" before the User is even aware of it.

Related Topic