Logging into text file or database

logging

When should I use database for logging and when text files?
I see that web servers and web frameworks (that your app uses internally) usually (always?) log requests and errors into text files by default. But I see that people who develop their app around those servers and frameworks sometimes log into database (even the main DB of the app, not some external one).
Also maybe there is a difference between debug logs and audit logs — I have read this classification somewhere on this site.

Best Answer

In very general terms, logging to a text file is much faster than logging to a database. That's the main aspect of logging you need to consider.

The reason you log to a DB is more likely to be because you want to query the results - searching for particular log info is easier in a DB, particularly if you log contextual information that can be used to group log entries together. It is also usually easier to access a central DB than a log file on a server which may be secured and not accessible.

The ideal would be to log locally to a file, and then migrate this data to a DB for inspection if needed afterwards.

Now auditing is a different beast altogether. While it is similar in concept to logging, audit is usually needed to be kept for a long time (unlike log files used for debugging or tracing that may be deleted at a whim). Audits are there to show important information. You log far less audit information and less often than normal logging so performance is not a concern. Its for this reason that the advantages of writing this audit info to a central DB are seen.

Related Solutions

Java Logging – Rules and Advice for Effective Logging

As an extension of your rule of logging where in the application the log statement came from, you may want to add per module level logging flags. Instead of logging everything, all the time, this allows you to selectively target sections of your application instead. There is overhead in this, and you need to create a facility that allows you to enable / disable that logging. Ideally, you would be able to enable / disable on-the-fly as the application is running.

I'm used to seeing a layer below debug which I term "Trace", but that's not necessarily a universal term. "Trace" level logging tracks as much as you can possibly stand, including module entry / exit, timestamps with entry / exit, and bonus points for capturing passed values. Obviously, that generates a LOT of data and it's not something you turn on willy-nilly. But it has advantages with respect to debugging when you can't attach to the process or you don't have a core dump of the errant application.

I like to see file / module references and timestamps with my log information. It can be handy when trying to hunt down race conditions between threads as well as coordinating the activities of multiple areas of the application. To be fair, I know of some folk who think these details clutter the log file. Adding in the timestamp is something to discuss with the team. (Apologies if log4j already does that.)

If the logging isn't being taken care of by it's own thread / process, that's something to consider as well. Instead of making the application thread wait for the logging to process, the log message gets passed off to the log handler and the application thread goes on its merry way. Alternatively, creating some sort of buffer mechanism to handle the log messages is another way to speed up application responsiveness.

Having controls on the size and history of log files is another feature to consider. You don't want the app blowing out all the disk space on the host system, nor do you necessarily want to keep all log files for all eternity.

Java Logging – Printing to Out vs Logging

The fact that you are writing a console application doesn't change good UI practices.

Your business logic should be distinct from your UI. The business logic performs the operations and indicates what might have gone wrong. The UI decided how that should be displayed.
If there is information that is useful for tracking the execution of the application after something has happened, it should be written to a log file. This will primarily happen in your business logic classes. The fact that something is written to a log file does not inherently mean that the UI does not need to display it. Things that are important enough to display to the user is likely important enough to log, but it does not need to be in the same form.

What goes in the log should be more detailed: What exactly went wrong? What were the inputs when that happened? You can also give more fine grained details about what is currently executing. To the user, it might be a single task, but to the code it could be a four step process. The nice thing about logging is the different levels let you have very detailed log messages in your code, but allowing some of them to be omitted from the log file.
Don't write code that is statically configured to only work one way. A command line application will obviously be dealing with standard out a lot. However, that doesn't mean your classes must have System.out.println() everywhere. You classes could write to an OutputStream. By initializing your UI classes with the OutputStreams/InputStreams that they should work with, it will make it much easier to test the code. Being able to use a ByteArrayOutputStream in an automated test is a great way to check if content is being output correctly.

An example of what I am suggesting:
```
public class A {
  private final OutputStream _out;

  public A(OutputStream out) {
    _out = out;
  }

  public void Run() {
    _out.println("Write to UI here");
  }
}
```
It is true that System.out is an OutputStream, but there are many classes that implement that interface. If you hard code your classes to use System.out, that is the only way the code can be used. If you pass in the output stream to write to, now you have the option to write to files, network sockets, and other things.

Best Answer

Related Solutions

Java Logging – Rules and Advice for Effective Logging

Java Logging – Printing to Out vs Logging

Related Topic