Linux – Why does the “nocto” NFS mount option in Linux not prevent flush-on-close

arch-linuxlinuxnfs

I've been learning about the close-to-open policy of NFS, which causes each
file to be flushed to the server as it's closed, to ensure consistency across
clients. (See http://docstore.mik.ua/orelly/networking_2ndEd/nfs/ch07_04.htm.) This leads to a big performance hit when trying to write many small files.

I know about the "async" export option, obviously, but there is also a "nocto"
client mount option, which is supposed to disable the close-to-open mechanism
for that client. As far as I can tell, this is supposed to stop the client
from flushing the file when it closes (as well as not bothering to check
cache consistency when it opens). However, this seems to have no effect: the
client is still flushing the file to the server on close, causing massive
wait-io.

Does anyone have any idea why "nocto" doesn't have the effect I was hoping it
would? The "async" option works as expected, but it's more important to me
that the client cache is correct in this case, and it just bugs me.

Example: a set of diskless nodes share a remote root, which is occasionally updated from one of the nodes. It is not critical that each file is flushed immediately after it is closed, as no other node is trying to write to the same file. However, it is more critical that if the server crashes whilst updating a set of packages, the client is aware of what data has not yet been written to the server's disk, so that it can try again once the server is up again. Using the "async" option, this scenario might result in data loss (because the server lies to the client about data being flushed to disk), whereas disabling close-to-open (and using "sync" instead of "async") should in theory provide the same performance benefit without the potential data loss (because several file writes would be buffered and flushed to the server together). The server and other clients would see a slightly outdated view of the filesystem (by a couple of seconds). This seems reasonable to me.

To put it really simply, "async" does server-side buffering, which speeds things up a lot. What I'm expecting is that "nocto" should do client-side buffering, with similar speed boosts, at the cost of some lag in data appearing on other clients.

Best Answer

From the nfs(5) man page:

If the nocto option is specified, the client uses a non-standard heuristic to determine when files on the server have changed.

Using the nocto option may improve performance for read-only mounts, but should be used only if the data on the server changes only occasionally. The DATA AND METADATA COHERENCE section discusses the behavior of this option in more detail.