Linux – Can XFS or GlusterFS replace the ZFS needs

centosglusterfslinuxxfszfs

Right now I have a FreeBSD hosts with ZFS and NFSv4. It is replicated to another FreeBSD box for backup purposes.

The ZFS features that are important to me are

  • software RAID6
  • snapshots, or some other way of replication to another host
  • quota
  • ACL
  • replace a failed disk without taking the host offline

Question

Could the same or similar setup be done with XFS or GlusterFS on CentOS 6?

Update

The hardware is

  • Supermicro CSE-847E16-R1400LPB chassis, 36 HS bays
  • Supermicro H8DG6-F AMD Dual G34 mainboard
  • AMD Opteron 6320, 2.8GHz 8-core, 8MB L2 cache, 6400MT
  • 64GB ram, and 128GB swap

Each host have 36*3TB space in RAIDZ2, so 100TB usable and 50TB used.

It seams that it is the Ubuntu clients that can crash the host on heavy reads. For now there are ~5 NFS clients. No read caching.

No NFSv4 tuning besides enabling Jumbo Frames

echo 'kern.ipc.nmbclusters="32768"' >> /boot/loader.conf

echo 'kern.ipc.maxsockbuf=16777216' >> /etc/sysctl.conf
echo 'net.inet.tcp.sendspace=262144' >> /etc/sysctl.conf
echo 'net.inet.tcp.recvspace=262144' >> /etc/sysctl.conf
echo 'net.inet.tcp.rfc1323=1' >> /etc/sysctl.conf
echo 'net.inet.tcp.sendbuf_max=16777216' >> /etc/sysctl.conf
echo 'net.inet.tcp.recvbuf_max=16777216' >> /etc/sysctl.conf

Best Answer

♡ Hey there...

I read this question as really being a problem with the FreeBSD NFS stack...

ZFS works very well on the supported platforms. So much so, that I've moved most of my ZFS systems running Solaris and NexentaStor to Linux (RHEL/CentOS), thanks to the ZFS on Linux project. If you're using ZFS now, going to anything else is a step backwards.

I'm curious about the following, though:

  • How much data are you storing?
  • How many NFS clients do you have?
  • Have you performed any NFS tuning on your existing servers?
  • Are you using any form of L2ARC read caching on the existing setup? How much RAM do you have?
  • What is the hardware configuration of your servers?

Regardless of the answers to the above, you have a few options...

  • Fix or debug your FreeBSD issue. NFS shouldn't crash servers. It may be worth getting to the root-cause of this problem if you have a lot of time invested in this setup.
  • Convert to ZFS on another platform. NexentaStor, Linux, Solaris, OpenIndiana are pretty solid on the NFS side.

All-in, some combination of XFS and a cluster filesystem can so some of the same things as ZFS, but it's not a direct comparison. I don't think you should abandon ZFS yet.

Related Topic