Linux – use syslog-ng to mirror all log files on a remote server without specifying every file

centoslinuxloggingrsyncsyslog-ng

Can I use syslog-ng to mirror all log files onto a remote server without specifying every file? Both are running syslog-ng OSE version 3.5.2. The file("/var/log/*") setting seems promising, but it does not appear to recurse, and reconstituting the file names on the other side of a syslog() source seems daunting.

Should I give up on this and do some sort of scripting or use rsync or something else entirely?

The source is essentially an embedded Linux computer with limited flash storage and the others is pretty hefty. They are connected to the same fast switch at 1GB. It is not OK for the embedded Linux to crash without having already sent most of its logs over, so some sort of continuous update would seem appropriate.

Best Answer

it depends.

If you have applications that log directly into files under /var/log/, then you have the following possibilities

  • Reconfigure the applications to log into syslog instead of files, then syslog-ng can read the incoming messages and forward them to your logserver. If your applications support logging into syslog, then this is the recommended way to go.
  • The commercial version of syslog-ng supports wildcard file sources (/var/log/* and the like). This feature is currently not available in syslog-ng Open Source Edition
  • As a workaround, you can use the confgen plugin of syslog-ng (I'm not sure it is available in version 3.5, but it surely is in 3.6 and newer). With the confgen plugin, you can run a script that generates a section into the syslog-ng configuration file: you can use it to list the files in /var/log as file sources (note that this solution will not add new files from /var/log automatically, you'll have to periodically restart syslog-ng for that).

BTW, if possible, you might want to update your syslog-ng to a newer version, 3.5 is rather old and somewhat buggy. The recent 3.9 version supports diskbuffers and other nice features.

HTH

Related Topic