It is quite common to test JSON / XML producing methods against file-stored expected output (at least in Java world, but probably in other environments, too).
For instance there's a method that produces RSS feed of new offers in the service, which has different formats (for different RSS consumers) and different options (query params, specific dates, short/long format, etc).
After a while there are 20-30 files that are validated against generated output in unit tests. When new request comes from the backlog a new file is created to meet the expectations and a new unit test is written.
When the output of the XML changes (a new field mandatory for all formats appears, or date format changes) it must be changed in all files stored for tests. Which makes files useless and painful to maintain in the long run.
My question is: how to mitigate that?
Should unit tests be written purely for fragment of the generated XML only? For instance separate unit tests for date field, separate for new mandatory field, etc?
How to test different outputs/formats/query params to the method producing XML? How tests should be organised to make sure the result XML is in a valid format, but at the same time change of a single property doesn't require changing all of the unit tests?
Best Answer
Tests of the kind you describe are brittle. As you've correctly pointed out, small changes are likely to break a significant number of the existing tests. But that is by design; tests like these are meant to break over the smallest change to your code. They are the proverbial "canary in the coal mine," meant to detect breaking changes in your code before your users do.
Such tests are relatively easy to create, and you shouldn't need a lot of them, because they cover a broad swath. All you have to do is look at the output of the code under test (given a particular input), and if you like it, paste it into the unit test assertion (or your files). This still holds true if you break your tests with new code; simply review all of the outputs, and re-paste them into your unit tests.
But such tests don't tell you anything about what might specifically have broken, only that something did, in fact, break. It's now up to you to figure out what changed, how it affected the output, and whether or not the new output is desirable.
The alternative is to find a way to break your code and your unit tests into smaller, more independent chunks, in such a way that, if you make a change to the code, it only breaks one or a few unit tests. To do that, you will have to invest a significant amount of time and effort into rewriting your code and your tests. Ultimately, this is probably a better, more maintainable approach.
Is it worth the trouble? Only you can answer that.