When to use delaycompress option in logrotate

logrotate

The man page of logrotate says that:

It can be used when some program cannot be told to close its logfile
and  thus  might  continue writing to the previous log file for some
time.

I'm confused by this. If a program cannot be told to close its logfile, it will continue to write forever, not for sometime. If the compression is postponed to next rotation cycle, the program continues to write to that file even after the next rotation cycle. How is postponing solving the problem?

My understanding is that copytruncate should be used when a program cannot be told to close the logfile. I'm aware that some data written to the logfile gets lost when the copy is in progress.

I was looking at the logrotate file for couchdb, and it had both copytruncate and delaycompress options.

/usr/local/couchdb-1.0.1/var/log/couchdb/*.log {
   weekly
   rotate 10
   copytruncate
   delaycompress
   compress
   notifempty
   missingok
}

It looks like there is no point using delaycompress when copytruncate is already there. What am I missing?

Best Answer

Your understanding of copytruncate is correct, but the wording in the manpage for delaycompress is a little misleading. More properly, it should say "when some program cannot be told to immediately close it's logfile" -- for instance, if you're using sharedscripts and the script sends a signal to the process using the log when all the log files have been rotated.

Related Solutions

Openvpn – Logfiles go blank after logrotate rotates them

OpenVPN is probably still writing to the OLD log file (the file (inode) that it opened for writing back when it started up).

Logrotate needs to notify the OpenVPN daemon that its log file has been turned over so OpenVPN can open the new logfile (This is usually done via a signal, but stopping/restarting the whole daemon has the same effect. Check the OpenVPN man page & your "postrotate" script).

Alternatively you can configure OpenVPN to use syslog, since logrotate should be notifying syslogd when it rolls your log files (or all your logfiles would be blank).

Logrotate Issue – Original File Size Restored After Rotation

This is probably because even though you truncate the file, the process writing to the file will continue writing at whichever offset it were at last. So what's happening is that logrotate truncates the file, size is zero, process writes to the file again, continuing at the offset it left off, and you now have a file with NULL-bytes up to the point where you truncated it plus the new entries written to the log.

od -c after truncate + sudden growth, generated output along the lines of:

0000000  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
*
33255657600  \0   C   K   B   -   s   e   r   v   e   r       [   h   t   t
33255657620 <more log output>

What this says is from offset 0 to 33255657600 your file consists of null bytes, and then some legible data. Getting to this state doesn't take the same amount of time it would take to actually write all those null-bytes. The ext{2,3,4} filesystems support something called sparse files, so if you seek past a region of a file that doesn't contain anything, that region will be assumed to contain null-bytes and won't take up space on disk. Those null bytes won't actually be written, just assumed to be there, hence the time it takes to go to 0 to 3.5GB don't take a lot of time to. (You can test the amount of time it takes by doing something like dd if=${HOME}/.bashrc of=largefile.bin seek=3432343264 bs=1, this should generate a file of over 3GB in a few milliseconds).

If you run ls -ls on your logfiles after they've been truncated and had a sudden growth again, it should now report a number at the beginning of the line which represents the actual size (in blocks occupied on disk), which probably is orders of magnitude smaller than the size reported by just ls -l.

Related Topic