Design – Multi-Layered Architecture: where I should implement the error logging \ handling

designerror handling

I'm currently refactoring a large subsystem with a multi layered architecture, and I'm struggling to design an effective error logging \ handling strategy.

Let's say that my architecture consists of the following three layers:

  • Public Interface (I.E an MVC Controller)
  • Domain Layer
  • Data Access Layer

My source of confusion is where I should implement the error logging \ handling:

  1. The easiest solution would be to implement the logging at the top level (I.E the Public Interface \ MVC Controller). However this feels wrong because it means bubbling up the exception through the different layers, and then logging it; rather than logging the exception at it's source.

  2. Logging the exception at it's source is obviously the best solution because I have the most information. My problem with this is that I can't catch every exception at source without catching ALL exceptions, and in the domain / public interface layer, this will lead to catching exceptions that have already been caught, logged and re-thrown by the layer below.

  3. Another possible strategy is a mix of #1 and #2; whereby I catch specific exceptions at the layer they are most likely to be thrown (I.E Catching, logging and re-throwing SqlExceptions in the Data Access Layer) and then log any further uncaught exceptions at the top level. However this would also require me to catch and relog every exception at the top level, because I can't distinguish between errors that have already been logged \ handled vs those that have not.

Now, obviously this is an issue in most software applications, so there must be a standard solution to this problem that results in exceptions being caught at source, and logged once; however I just can't see how to do this myself.

Note, the title of this question is very similar to 'Logging exceptions in a multi tier application"', however the answers in that post are lacking detail and are not sufficient to answer my question.

Best Answer

To your questions:

The easiest solution would be to implement the logging at the top level

Having the exception bubble up to the top level is an absolutely correct and plausible approach. None of the higher layer methods tries to continue some process after the failure, which typically can't succeed. And a well-equipped exception contains all the information necessary for logging. And doing nothing about exceptions helps you keep your code clean and focussed on the main task instead of the failures.

Logging the exception at it's source is obviously the best solution because I have the most information.

That's half correct. Yes, most useful information is available there. But I'd recommend to put all this into the exception object (if it isn't already there) instead of immediately logging it. If you log at a low level, you still need to throw an exception up to tell your callers that you didn't complete your job. This ends up in multiple logs of the same event.

Exceptions

My main guideline is to catch and log exceptions at the top level only. And all the layers below should make sure that all necessary failure information gets transported up to the top level. Within a single-process application e.g. in Java, this mostly means not to try/catch or log at all outside the top level.

Sometimes, you want some context information included in the exception log that's not available in the original exception, e.g. the SQL statement and parameters that were executed when the exception was thrown. Then you can catch the original exception and re-throw a new one, containing the original one plus the context.

Of course, real life sometimes interferes:

  • In Java, sometimes you have to catch an exception and wrap it into a different exception type just to obey some fixed method signatures. But if you re-throw an exception, make sure the re-thrown one contains all information needed for later logging.

  • If you are crossing an inter-process border, often you technically can't transfer the full exception object including the stack trace. And of course the connection might get lost. So here's a point where a service should log exceptions and then try its best to transmit as much of the failure information as possible across the line to its client. The service must make sure the client gets a failure notice, either by receiving a failure response or by running into a timeout in case of a broken connection. This will typically result in the same failure to be logged twice, once inside the service (with more detail) and once in the client's top level.

Logging

I'm adding some sentences about logging in general, not only exception-logging.

Besides exceptional situations, you want important activities of your application to get recorded in the log as well. So, use a logging framework.

Be careful about log levels (reading logs where debug information and serious errors are not flagged with different accordingly is a pain!). Typical log levels are:

  • ERROR: Some function failed irrecoverably. That doesn't necessarily mean your whole program crashed, but some task couldn't be completed. Typically, you have an exception object describing the failure.
  • WARNING: Something strange happened, but didn't cause any task to fail (strange configuration detected, temporary connection breakdown causing some retries, etc.)
  • INFO: You want to communicate some significant program action to the local system administrator (starting some service with its configuration and software version, importing data files into the database, users logging into the system, you get the idea...).
  • DEBUG: Things you as the developer want to see when you are debugging some issue (but you'll never know in advance what you really need in case of this or that specific bug - if you can foresee it, you'll fix the bug). One thing that's always useful is to log activities on external interfaces.

In production, set the log level to INFO. The results should be useful to a system administrator so he knows what's going on. Expect him to call you for assistance or bug-fixing for every ERROR in the log and half of the WARNINGs.

Enable the DEBUG level only during real debugging sessions.

Group the log entries into appropriate categories (e.g. by the fully-qualified classname of the code that generates the entry), allowing you to switch on debug logs for specific parts of your program.

Related Topic