Using rsyslog to create different log files for different processes

loggingrsyslogsyslog

Scenario: I am running a cluster of machines. Each machine runs various python programs with a unique (across the cluster), but dynamically set, ID.

Right now, they are all logging locally. So, I might have logs that look like:

process_5.log
process_6.log

for processes that had ID's 5 and 6.

Another machine may have:

process_20.log
process_25.log

I wish to forward these logs to a logserver running rsyslogd. Python's logging facility has a nice syslog handler, so I understand how I could connect to the remote server. What I haven't figured out is how to use templating/DynFile to maintain log separation.

e.g. on the logserver, I will want to see:

process_5.log
process_6.log
process_20.log
process_25.log

which correspond to the logs of the same name on the sending machine.

Is there a way to pull this off with rsyslogd templating?

Best Answer

As the user quanta posted, their template should work. Here is mine:

$template DynFile, "/var/log/syslog/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfacility-text%.log"
*.info;mail.none;authpriv.none;cron.none            ?DynFile
authpriv.*                                          ?DynFile
mail.*                                              ?DynFile
cron.*                                              ?DynFile
uucp,news.crit                                      ?DynFile

This will make logs for each app or facility go into its own file. Each client's log on the server will appear just like on the client.

Notice the %syslogfacility-text%. You can substitute this part to various other properties per the Property Replacer documentation.

Related Topic