Linux – How to move /var/log directory

debiandisk-space-utilizationlinuxlog-files

On our production server there is a small drive for the root mount point /,
/var/log is taking too much space and I have to manually delete some files.
How can I move /var/log/ to let's say /home/log WITHOUT REBOOTING?

Here is the thing I thought:

$ mkdir /home/log
$ rsync -a /var/log /home/log
$ mount --bind /home/log /var/log
$ /etc/init.d/rsyslof restart

But I know that some services use file descriptors, so they'll continue to use /var/log or inodes.

Best Answer

Proper design

I assume you are unable to simply extend the filesystem in question (using lvextend && ext2online), because you do not use LVM or use wrong filesystem type.

Your approach

What you've proposed might work if you signal the daemons with SIGHUP (kill -1 pid). Obviously you would need to later on "mount -o bind / /somewhere" and clean up what has been left underneath mounted /var/log. But it has a bad smell for me, especially for production.

Avoid downtime, have a clean result (but complicated to do)

Forget about "mount -o bind" idea, create a new LV/partition, but don't mount it yet.

lsof | grep /var/log             # lists open files in /var/log

For each daemon that has any open file (I would expect at least syslog, inetd, sshd):

  • reconfigure the daemon no to log to /var/log
  • refresh the daemon (kill -1 or /etc/init.d/script reload)
  • confirm with lsof | grep /var/log that daemon has closed its files

Mount over /var/log. Restore old configurations, SIGHUP/reload daemons again.

Easy way (downtime)

Create a new LV/partition and mount it properly over either /var or /var/log. The easy way is to take down the server to maintenance mode (single-user mode), and use the actual console (not ssh) for the operation.