I keep it simple.
A library has a base exception type extended from std:::runtime_error (that's from C++ apply as appropriate to other languages). This exception takes a message string so we can log; every throw point has a unique message (usually with a unique ID).
That's about it.
Note 1: In the situations where somebody catching the exception can fix the exceptions and re-start the action. I will add derived exceptions for things that can be potentially uniquely be fixed at a remote location. But this is very very rare (Remember the catcher is unlikely to be close to the throw point thus fixing the problem is going to be hard (but everything is dependent on situation)).
Note 2: Sometimes the library is so simple it is not worth giving it its own exception and std::runtime_error will do. It is only important to have an exception if the ability to distinguish it from std::runtime_error can give the user enough information to do something with it.
Note 3: Within a class I usually prefer error codes (but these will never escape across the public API of my class).
Looking at your trade offs:
The trade-offs I see include:
More exception classes can allow very fine grain levels of error handling for API users (prone to user configuration or data errors, or files not being found)
Do more exceptions really give you finer grain control? The question becomes can the catching code really fix the error based on the exception. I am sure there are situations like that and in these cases you should have another exception. But all the exceptions you have listed above the only useful correction is to generate a big warning and stop the application.
More exception classes allows error specific information to be embedded in the exception, rather than just a string message or error code
This is great reason for using exceptions. But the information must be useful to the person who is caching it. Can they use the information to perform some corrective action? If the object is internal to your library and can not be used to influence any of the API then the information is useless. You need to be very specific that the information thrown has a useful value to the person that can catch it. The person catching it is usually outside your public API so tailor your information so that it can be used with things in your public API.
If all they can do is log the exception then it is best to just throw an error message rather than lots of data. As the catcher will usually build an error message with the data. If you build the error message then it will be consistent across all catchers, if you allow the catcher to build the error message you could get the same error reported differently depending on who is calling and catching.
Less exceptions, but embedding an error code that can be used as a lookup
You have to determine weather the error code can be used meaningfully. If it can then you should have its own exception. Otherwise your users now need to implement switch statements inside there catch (which defeats the whole point of having catch automatically handle stuff).
If it can't then why not use an error message in the exception (no need to split the code and the message it makes it a pain to look up).
Returning error codes and flags directly from functions (sometimes not possible from threads)
Returning error codes is great internally. It allows you to fix bugs there and then and you have to make sure you fix all error codes and account for them. But leaking them across your public API is a bad idea. The problem is that programmers often forget to check for error states (at least with an exception an unchecked error will force the application to quit an un-handled error will generally corrupt all your data).
Implemented an event or callback system upon error (avoids stack unwinding)
This method is often used in conjunction with other error handling mechanism (not as an alternative). Think of your windows program. A user initiates an action by selecting a menu item. This generates an action on the event queue. The event queue eventually assigns a thread to handle the action. The thread is supposed to handle the action and eventually return to the thread pool and await another task. Here an exception must be caught at the base by the thread tasked with the job. The result of catching the exception will usually result in an event being generated for the main loop which will eventually result in an error message being displayed to the user.
But unless you can continue in the face of the exception the stack is going to unwind (for the thread at least).
In short, the only time you should log the existence of an exception is when you are handling it.
When you throw an exception, it is because your code has reached a state where it can not proceed correctly. By throwing an exception, you are representing a specific message to your program about the error that occurred. You should not catch an exception until you are at a point where it can properly be handled.
The code you write as part of your main application should be aware of the types of exceptions that can be thrown and when they might be thrown. If you can't do anything productive with an exception, don't catch it. Do not log an exception until it is being handled. Only the handling code knows what the exception means in the context of the program flow and how to respond to it. Writing a log message here may make sense here. If you use a logging framework, you can set a log level for the message and potentially filter them. This works well for exceptions that can occur, but are not critical and can be recovered from cleanly.
Your exception catch-all, is your last ditch effort to keep your code from crashing an ugly death. If you have gotten this far, you log all the state and error information you can. Then you do your best to nicely tell the user that the program is crashing before everything stops. Your goal should be to never have this code executed.
Embedding logging into the base class does not follow the above guidelines. The base class doesn't know anything about the state of the code. (Having the stack trace doesn't count because you are not going to write code that makes decisions based on parsing it.) The base class can't do anything to indicate the severity or how the exception might be handled. You don't want huge data dumps and stack traces every time there is a simple exception that you can handle and recover from cleanly.
Best Answer
Firstly, I didn't understand what you need the monitor thread for and what is it's purpose. The only reason I could think of is if the monitor thread is a continually running thread that just watches the buffer and periodically calls the database write. If that is the purpose, I would rather do it with schedulers, e.g. schedule a write every n seconds and write the whole buffer to the database. Then you just need to approximate how much seconds you need the scheduler rate to be in order for the buffer not to overflow.
As for the exception handling in threads, I would first retry the write. Ultimately, if everything fails, first thing you should do is salvage the data so it doesn't get lost.
The way exceptions are meant to be handled is to identify the exceptions that can occur and then only handle those specific exceptions. You should never do things like
catch (Exception e). Since the DB writer service is a general service, I would have it propagate the exception to the caller and then have the caller handle the exception, since the service cannot know what behavior the caller wants.As for how you should handle those specific exceptions is up to you and hardly anyone can help you with that. If you cannot handle the exceptions in
finallyblock and they need developer's attention, I would log them or notify the developer through alternate communication channel, e.g. email, Slack.