Let me explain what I mean.
I have made a complex, highly polished over years PHP framework/library for my own use. I very aggressively log the smallest notice and immediately deal with it as soon as it pops ups, always trying to predict potential errors in my code as to never have them occur even in rare situations, but rather handling them automatically before they get logged.
However, in spite of all my efforts, inevitably, I wake up (such as today) to find that some third-party service has fiddles around with their file format for one of their CSV files of data that they provide on their website and which my system fetches and imports every day.
Then I get a flood of ugly PHP errors. Ouch.
Even though it looks scary at first, it's typically just a pretty simple fix, and it's typically really just ONE error, which cascades into tons of apparent errors because the chain of function calls "fall apart" as each one expects something that they no longer get.
I fix the issue, clear the errors, re-run the logic, verify that it no longer causes any errors, and then it's fixed. For now. Until the same thing happens again, with some other part of the system.
I can personally "deal with" this, but it really bothers me in terms of giving away my system to somebody else to run on their machines. If/when the same kind of thing happens for them, they will doubtlessly blame me and think I'm incompetent (which may be true).
But even for myself, this is quite annoying and makes me feel as if my system is very fragile and a house of cards waiting to fall apart, in spite of there normally being not a single little notice or warning logged during "normal operation".
Short of predicting every possible change and writing enormous amounts of extra "checking" code to verify that all data is always exactly what is expected, is there anything I can do to fix this general problem? Or is this like asking for a pill that cures any disease instantly?
Please don't get hung up on the fact that I mentioned PHP. I'd say that this question goes completely regardless of the programming language or environment. It's really more of a philosophical question than a technical one IMO.
I fear that the answer will be: "There is no way. You have to bite the bullet and verify, verify and verify everything all the time!"
Best Answer
An improvement would be to design your system to fail gracefully. If the first step of parsing a file fails, then stop with an error. Don't carry on passing bad data from one step to the next.
The other thing to check is that you are implementing the file handling correctly and robustly. CSV is quite complicated when you encounter quoted strings with embedded commas in them. If the supplier has actually changed the file format, then you should stop processing. If they have used a feature of CSV that you haven't implemented right, you need to fix that robustly.