I have a machine running FreeBSD that is subject to a variety of workloads, rabging from Disk intensive to memory intensive.
If large amounts of ZFS data are accessed, then the Wired Memory usage can quickly grow to 95GB where there is only 96GB installed, resulting in process crashes and an unresponsive machine.
The checksetuid and neggrpperp processes are capable of trigering this on their own.
as a work around, setting vfs.zfs.arc_max to around half the system ram prevents process crashes however this also has the possible performance issue that at times it would be optimal if the system could allocate more RAM to the arc when it is not needed for other processes.
When ram is needed for other processes it seems that the wired memory usage drops excessively slowly.
security_status_chksetuid_enable and security_status_neggrpperm_enable are also set to NO so as not to fill the arc with useless stuff.
Fundamentally, is there a better way of handeling this other than setting vfs.zfs.arc_max to a slightly arbitrary value ?
Best Answer
The ZFS ARC is a subject to automatic scaling in any system using ZFS. ARC limiting is only a workaround; the behavior you’re experiencing was pretty much usual for FreeBSD older than 11.x (however, I personally haven’t seen any crashes -only ineffective memory usage); after 11.0 release this behavior was massively mitigated (thus, if you’re using 10.x or even 9.x please upgrade). If you’re still seeing this on a recent 12.x, please post a message in freebsd-stable@ ML or file a comprehensive PR. I don’t think that filing an issue ticket to the Open-ZFS bugtracker or ML is appropriate. You could also use the
sysutils/zfs-statsport to narrow down what’s happening inside the kernel part of the memory and the ARC itself, you will be probably asked to attach the result when consulting the stable@ ML.