We use Log4J with its SyslogAppender to send messages to a central syslog-ng server, all running on Unix machines.
Is there a way (whether that's in Java or in Unix) to throttle the number of messages that are sent in order to avoid an upset server also upsetting the network?
The only option I can think of is to set the log level higher so less messages will actually be sent, but that isn't ideal as a important messages could be suppressed on a machine that is otherwise behaving itself.
I suppose in an ideal world a dynamically-changing level would be good: if the number of messages/second passes a certain level, the threshold rises, but at the same time that sounds a bit like overkill.
Any ideas?
Thanks
Rich
Best Answer
I think the question is, do you care if you don't get all your logs on the central server? What you're talking about is essentially dropping messages--in which case, you'll lose logs. Is this okay? If it is, you've already answered your own question--raise the debugging level to only get messages you really care about.
If, however, you're trying to match, say, a bandwidth constraint (such as Splunk's month processing limit), you'll need to write an intermediary server to take the logs from syslog and prioritize them. Its not difficult, but it is highly specific to your use case. One bonus with this method is that this middleman can immediately send important logs to the aggregation server and, at the end of the day/month, send the next highest priority logs that weren't sent originally. That way, you can fill the quota exactly.
If you add more specific requirements (such as why you need to do this), and what you mean by limiting logs (duplicate lines? bandwidth? space? aggregation server can't keep up? etc..) then you'll get a much better answer.
Good luck!