Web Development – Server-Side Event Logs: DB vs File Writing

databaselanguage-agnosticweb-development

I am developing a web application as well as the API the web application uses. I'm trying to determine whether it is better to log events (to determine the path that leads to an error as well as to determine whether the site has been compromised) in a database or simply by writing to a file.

On the one hand, I don't want to bog down my database with a queries every single time I need to log an error. On the other hand, I don't want to have to dig through a giant event / error log file. I'm thinking of turning into re-write mode once the file / table reaches 10,000 or 100,000 events.

I know the general factors I should consider:

  1. Performance (both DB and general–in my case I'm using PHP and Postgres)
  2. Ease of finding the path a user took to create a bug or error (I'm almost certain DB is better for this)
  3. Scalability – Is this the same solution I'm going to be using with 20 users as I am with 100,000 users?

Can you tell me how these two possible solutions fit in with the above–really, I think the most important one for me is performance. How demanding is it to write to a file vs. writing to a DB and if every single event and command sent by the user (and there will be many) is logged (before a re-write limit), which solution will end up being faster?

Best Answer

If logging to the DB, where would you log db failures, like the DB being unavailable?

Yes, if logging to the file system, the file system could become unavailable, but my guess is that you then would have other things to worry about...

In a server application (not a web server, but one under heavy load anyway), we log to files (several), using a specialist logging component. And while performance is a crucial factor in many parts of the code, we have yet to feel the need to decrease our logging because of performance considerations.

Probably because most logging is done outside of the time critical code. Which is only "natural" because time critical code often would produce way too many log messages to be of any practical use.

Related Topic