I have a controller server, and two 24 SAS fileserver arrays. Each fileserver array is set up as a ZFS pool with 4 RAIDZ2 vdevs with 6 drives each.
Ultimately, I want to be able to use the controller server as a single mount point for the two fileservers. The main idea being that my end users only have to access the one controller server and one mount point in order to access / back up their data across the combined storage capacity of two servers. I was thinking of using glusterfs for this, but the information I've been able to find so far seems to be for generation of redundant storage pools. In this application, gluster is being used as a distributed mirror RAID1, which doesn't really fit my need as it would create another level of unneeded redundancy.
How would you suggest I go about creating a non-redundant distributed filesystem from the two ZFS pools hosted on separate computers? Is there a way to accomplish this with gluster, or if there a different NFS that better suited to accomplishing this? I've considered samba as well, but I'm not sure if it will be secure enough. I genuinely liked the prospect of only having ssh publicly hosted on the controller with strict 2FA authentication.
Best Answer
You'll need to make the choice as to whether you want a distributed or striped volume.
Distributed volumes are simple, and will hash out any files written to each node in as balanced a way as possible. It does nothing to the files themselves, and you'll see intact files on your ZFS "brick". Striped volumes will cut up your files and distribute them between nodes as chunks. This is a desireable configuration if your content is almost exclusively large files (such as videos, disk images, and backups).
You're just about on the nose thus far, as the command to create such a distributed volume is as simple as you've stated:
gluster volume create test-volume server1:/exp1 server2:/exp2 server3:/exp3 server4:/exp4While a striped volume is created thusly:
gluster volume create test-volume stripe 4 server1:/exp1 server2:/exp2 server3:/exp3 server4:/exp4HOWEVER, four stripes can cause a lot of overhead when grabbing a file. One would only do that if they were working with truly massive files. For a good compromise that will still work for large disk images and the like, I would suggest a distributed striped volume:
gluster volume create test-volume stripe 2 server1:/exp1 server2:/exp2 server3:/exp3 server4:/exp4This volume will stripe across two nodes, and distribute striped sets across another two nodes.