How does logrotate deal with copytruncate

logrotate

I understand that copytruncate is not an ideal setup, but that's the cards I'm dealt with…

I have a service that is tailing data from journalctl and streaming it to a file.

The setup to rotate the file is…

/var/log/xxxxxx/xxxxxx.log {
    size 100M
    copytruncate
    rotate 5
    compress
    compresscmd /bin/xz
}

When I look at the location /var/log/xxxxxx/, I see that xxxxxx.log is 8GB but I also see xxxxxx.log-20181125.gz and so on… which means that rotation is happening.

The question I have is shouldn't the main xxxxxx.log be at 100MB why is it 8GB? And also if it's 8GB which cause the logrotate rule to fire, does this means that it's also constantly trying to process that 8GB file so essentially consuming resources for nothing?

UPDATE
There is a service that runs: /bin/sh -c '/bin/journalctl --since="5 minutes ago" --no-tail --follow --unit="xxxxxx*.service" >> /var/log/xxxxxx/xxxxxx.log 2>&1'

When running logrotate -v /etc/logrotate.d/xxxxxx logrotate indicates that it truncated the file, but in fact it did not. I stopped the service and manually ran it again and it worked.

Running: CentOS Linux release 7.5.1804 (Core)

Best Answer

Try running a tool like wc to see how much data is in the file. You may be getting a sparse file with the next record at the file position just after the position where the file was when it truncated and nothing before that. Some services keep their file position and write at that position. This does not work well when using copytruncate to rotate the log file.

Some applications reopen their files when sent a signal, and it is common to allow logrotate to move and recreate the file as log data will continue to be written to the log file until it is reopened. Use delaycompress when using this method. The logrotation specification for the syslog daemon is a good example of this.

See the man page for logrotate for a discussion on rotating logs.

Related Topic