There is much debate as to whether FreeNAS can be run as a virtual machine.
The official position is that yes, but there is additional configuration required.
If I cannot guarantee that I can follow these recommendations, am I more vulnerable to failure – especially catastrophic failure – than if I run a vanilla Linux system with EXT4/XFS, or FreeBSD with UFS?
Specifically, assume I won't be able to do PCI passthrough nor can I disable write caching. Furthermore, I will only have one vDisk for storage (A VMDK backed by hardware RAID) so no RAIDZ. Obviously, there will be backups.
EDIT: To clarify why I want to do this – I need a file server, and this is the infrastructure I have to work with. If I need to I could get the extra vDisks to setup RAIDZ, but otherwise that's it. I was looking for a good file server solution, and FreeNAS seemed to fit the bill. Except that there were all of these dire warnings about virtualizing ZFS and how you can lose all of your data and corrupt your backups.
I realize that deploying FreeNAS on this infrastructure is risky. My question is: is it riskier than the alternatives?
EDIT2: I seem to be having trouble communicating my intent. FreeNAS with ZFS is a rock solid NAS platform. However, from what I have read, it seems that the features that make ZFS more reliable as a bare metal file server can actually work against you if you run it on a standard VM configuration. If so, then using a different file system is better choice on standard VM settings (i.e. no direct IO, write cache enabled). Is this a correct assessment?
Best Answer
General answer
The risks are different and not directly comparable.
send/recv
or snapshots make your live much easier and have nothing to do with integrity.Specific to your case
If you look at the referenced recommendations and dissect them, you notice a few things:
Scrub just reads every block of the underlying vdevs and verifies their checksums. If your virtual disk does not cope with this, it is garbage and you should be concerned about it, not about ZFS. On the other hand, if your virtual disks are already checksummed on the SAN, your additional scrub will do nothing except cause additional I/O (it is useless).
This is good advice if you don't trust the hardware. The downside is considerably lower performance, of course. You may also have no control ofer the SAN settings, so you need to treat it as a cheap disk you bought from ebay and slapped into your system - anything can happen, at least in theory.
This is okay as general advice, but a bit of a nitpick. Assuming your SAN is bad, this will help you in certain cases (with much luck at least). Assuming your SAN is good, this does nothing and just costs you space and performance. It is much better in my opinion to make sure that the chain from physical disks to SAN to network to VM host to VM guest is equally good, so you don't have to do everything again in each layer.
FreeNAS vs others
A word about the FreeNAS recommendations - they are certainly okay as recommendations, that is, guidelines or tips for the general audience. If you follow them, you will not be worse off then otherwise, and might even be better off. Then again, they are stern-worded, as seems to be the usual tone in the FreeNAS community (judging from certain forum posters at least). I guess they just want to be on the safe side with that. I have always preferred the ZFS Best Practices guide, because it is worded pretty neutral and just presents facts, leaving it up to you to decide.
It's also interesting that according to the FreeNAS docs and forums, you will die a gruesome death if you dare to run a ZFS system for file services with less than a pitiful 4GB of RAM, while on the mailing lists of OmniOS (or SmartOS or illumos or Nexenta, I don't remember at the moment) people tested systems with 512MB of RAM and shared their suggestions how to configure them. All in all, it was more about knowledge of details and the choice was left to each person, instead of establishing rules that thou shalt follow.
Over time, this problem will also become less important and the recommendations will change, as more and more systems switch over to ZFS on normal desktop and server editions. Ubuntu has already done it, and others will surely follow. If in two or three years 80% of the distributions use ZFS or btrfs, most of them will run virtualized, so it is a moot point.