Linux – How to Filter Syslog Entries

I think the question is, do you care if you don't get all your logs on the central server? What you're talking about is essentially dropping messages--in which case, you'll lose logs. Is this okay? If it is, you've already answered your own question--raise the debugging level to only get messages you really care about.

If, however, you're trying to match, say, a bandwidth constraint (such as Splunk's month processing limit), you'll need to write an intermediary server to take the logs from syslog and prioritize them. Its not difficult, but it is highly specific to your use case. One bonus with this method is that this middleman can immediately send important logs to the aggregation server and, at the end of the day/month, send the next highest priority logs that weren't sent originally. That way, you can fill the quota exactly.

If you add more specific requirements (such as why you need to do this), and what you mean by limiting logs (duplicate lines? bandwidth? space? aggregation server can't keep up? etc..) then you'll get a much better answer.

Good luck!

DRBD is great.

The good things:

• It does a magnificent job at replicating data
• DRBD has in a couple of cases prevented disaster, where it has discovered that the volume was already mounted on the other node, which the raw volumes we get from a SAN are unable to tell us.
• Heartbeat already has great support for DRBD.

The challenges:

• Remember to monitor it properly, so you discover split brains when they happen - and can deal with it.
• DRBD can't be mounted on both servers without a cluster-enabled filesystem on top - I don't have any experience with that part.
• It is easy to "DOS" the servers by configuring DRBD to use all the available bandwidth for syncing disks. Just configure for lower throughput, and you're OK.

For mounting "the same" filesystem on several nodes, we keep going back to NFS, even though we keep testing various solutions for it. A setup I have no problem with having in production is NFS on top of EXT4 on top of DRBD. I wouldn't dare to do this with the database filesystems, but it's OK for the wwwroot.