I'm running an email server with OpenSMTPD and Dovecot on Linux and accessing emails using IMAP with a Thunderbird client. When I delete an email in Thunderbird, why doesn't disk space usage go down?
As an example, one user's mbox files are stored in /var/vmail/${domain}/$[user}/:
$ ls
Archives Drafts inbox Sent Spam TrainSpam Trash
I'm not sure if mbox files are sparse files, so instead of du
(which also shows the issue), I expect ls
will have the most accurate "effective" file sizes, so I added up all sizes of all files in this directory:
$ ls -al | grep vmail | awk '{print $5}' | paste -sd+ | bc
1119217444
Next, I go to Thunderbird, and delete a large email with attachment which shows a size of 1MB. Thunderbird sends it to the Deleted folder, then I go to the Deleted folder, delete it there, confirm the permanent deletion dialog and re-count the file sizes:
$ ls -al | grep vmail | awk '{print $5}' | paste -sd+ | bc
1119217443
So it went down 1 byte. Perhaps it's just marking it deleted? How to I actually get back the disk space? I understand this may be non-trivial since an mbox file is just a huge, flat file.
Best Answer
In MBOX format messages are stored in a single huge file one below the other, with a very simple structure:
Therefore, deleting a message from the middle of the file will cause rewriting the rest of the file, which might be bad for both performance and data integrity, as the file might became corrupted if the write is interrupted.
One solution is to flag the message as deleted rather than actually deleting it, as it requires only modifying one line while keeping rest of the file intact. This allows combining multiple deletions into a single operation, later.
MozillaZine's article on Compacting folders explains this from the Thunderbird perspective:
Dovecot's article on Mbox Mailbox Format explains how Dovecot handles the problems with MBOX format. The deletion is stored in
X-Status: D
header added to the message headers.