Linux – What are the typical methods used to scale up/out email storage servers

email-serverlinuxreplicationscalabilitystorage

What I've tried:

  • I have two email storage architectures. Old and new.

Old:

  • courier-imapds on several (18+) 1TB-storage servers.
  • If one of them show signs of running out of disk space, we migrate a few email accounts to another server.
  • the servers don't have replicas. no backups either.

New:

  • dovecot2 on a single huge server with 16TB (SATA) storage and a few SSDs
  • we store fresh mails on the SSDs and run a doveadm purge to move mails older than a day to the SATA disks
  • there is an identical server which has a max-15min-old rsync backup from the primary server
  • higher-ups/management wanted to pack in as much storage as possible per server in order to minimise the cost of SSDs per server
  • the rsync'ing is done because GlusterFS wasn't replicating well under that high small/random-IO.
  • scaling out was expected to be done with provisioning another pair of such huge servers
  • on facing disk-crunch issues like in the old architecture, manual moving of email accounts would be done.

Concerns/doubts:

  • I'm not convinced with the synchronously-replicated filesystem idea works well for heavy random/small-IO. GlusterFS isn't working for us yet, I'm not sure if there's another filesystem out there for this use case. The idea was to keep identical pairs and use DNS round-robin for email delivery and IMAP/POP3 access. And if one the servers went down for whatever reasons (planned/unplanned), we'd move the IP to the other server in the pair.
  • In filesystems like Lustre, I get the advantage of a single namespace whereby I do not have to worry about manually migrating accounts around and updating MAILHOME paths and other metadata/data.

Questions:

  • What are the typical methods used to scale up/out with the traditional software (courier-imapd / dovecot)?
  • Do traditional software that store on a locally mounted filesystem pose a roadblock to scale out with minimal "problems"? Does one have to re-write (parts of) these to work with an object-storage of some sort – such as OpenStack object storage?

Best Answer

What I've seen with medium to large companies is redundant storage devices such as NetApp or EMC. In fact I was talking to an EMC rep about email storage a little while ago, and he said huge email servers are a very common sale for them.

Basically they take all of the storage issues away from the application. Performance for a lot of short random reads is achieved with SSD or battery backed memory cache. All the storage is in one place with multiple paths to redundant server modules, so there's no replication latency.

The application servers access the storage using NFS or iSCSI which is less flexible but sometimes required by the application not behaving well with NFS. That allows the storage to be shared by any number of servers over high speed ethernet, so you can scale to the maximum I/O performance of the storage box which you can then expand as needed.

As far as redundancy on the application servers, the cheapest is a software clustering package. There's also appliances like Big-IP that handle it at the network level and are OS independent. It depends heavily if the application can work reliably over NFS in parallel with other instances.

Related Topic