I have logs in /var/log/syslog/YYYY-MM-DD/file.log files. New directories are created automatically by rsyslog, and logs are automatically put to new locations.
Every day, I archive previous day of logs to another directory, so, aside from 5 minute period, there is only one subdirectory in /var/log/syslog/.
Based on this, I made fail2ban rules using:
logpath = /var/log/syslog/*/auth.log
Which works. Until midnight, where fail2ban still "listens" to previous log. I can reload fail2ban configuration, but this makes me lose existing bans.
So the question is – what can be done to make fail2ban to switch to new log, without losing existing bans?
I also tried to make symlink /var/log/syslog/today, to point to current day logs, and point fail2ban to /var/log/syslog/today/auth.log, but then I got in logs:
Jan 8 00:05:46 xxxx fail2ban.filter : ERROR Unable to open /var/log/syslog/today/auth.log
Jan 8 00:05:46 xxxx fail2ban.filter : ERROR [Errno 2] No such file or directory: '/var/log/syslog/today/auth.log'
Traceback (most recent call last):
File "/usr/share/fail2ban/server/filter.py", line 491, in getFailures
has_content = container.open()
File "/usr/share/fail2ban/server/filter.py", line 569, in open
self.__handler = open(self.__filename)
IOError: [Errno 2] No such file or directory: '/var/log/syslog/today/auth.log'
It could be because the auth.log didn't get created until couple of minutes later.
But it any way – fail2ban logged this error coupld of times, and then didn't monitor anything (didn't ban anything)
Best Answer
Edit When you start fail2ban, it opens filehandles to all the auth files it currently sees. It doesn't keep scanning the directories to see if any new files show up. Using a symlink doesn't work, because the symlink is to the old inode, not to the new one.
The simplest solution is to add a second, separate authlog file that's used only by fail2ban. You can empty it every midnight to avoid it getting too large.
Original answer The problem is that fail2ban opens the inode/filehandle belonging to that filename the first time you start it. It doesn't check for changes once the file is opened, so when the file is moved, its filehandle stops working.
The solution is to not move the file at all. Instead, copy its contents to the new file and then empty the old one. You've not said how exactly the rotation is performed, but most logrotation packages will have an option to copy rather than move.