Ubuntu – Monitoring important log files on multiple linux hosts

loggingmonitoringsystem-monitoringUbuntu

I have a few servers running on AWS and have Nagios/Icinga doing the monitoring of all critical services.

We're trying to figure out the best way to monitor all logs – system, DB, PHP, Apache, etc – on the system so we know about issues (for e.g. that Apache reached the max_clients threshold yesterday) immediately via email. We only look at logs currently after a service goes down, not before, which is bad.

I'm new to Linux administration and I've identified the following options after a search online:

  1. Nagios scripts to monitor logs – The problem is most of them check one log file for one specific regex at a time. It's not scalable to install one service for each log file (I don't even know all the log files we have to monitor!)
  2. A service such as logrobot.com – I'm not sure how effective this is though.

Appreciate your advice on what's the best way to monitor all these logs on multiple servers with minimal configuration.

Best Answer

Don't know how much servers/logs you have to monitor but there are many solutions out there

small environment

Use rsyslog and a frontend you like (ex. LogAnalyzer http://loganalyzer.adiscon.com/)

bigger environment

We monitor our serverlogs from (+300 system) with beaver as logshipper, logstash as indexer and elasticsearch as backend. This solution scales up to [insert random number here] hosts ;)

Related Topic