Nfs – Default rsize and wsize for nfs v3

When a client request comes in, it gets handed off to one of the threads and the rest of the threads are asked to do read-ahead operations. The fastest way to read a file is to have one thread do it sequentially... So for one file this is overkill, and the threads are in essence making more work for themselves. But what's true for 1 client reading 1 file won't necessarily be true when you deploy in the real world, so stick with the formula for basing number of threads and number of read-aheads off bandwidth/cpu specs.

Maybe it's not slower transfer speed, but increased write latency. Try mounting the NFS share async instead of sync and see if that closes the speed gap. When you rsync over ssh, the remote rsync process writes asynchronously (quickly). But when writing to the synchronously mounted nfs share, the writes aren't confirmed immediately: the NFS server waits until they've hit disk (or more likely the controller cache) before sending confirmation to the NFS client that the write was successful.

If 'async' fixes your problem, be aware that if something happens to the NFS server mid-write you very well might end up with inconsistent data on disk. As long as this NFS mount isn't the primary storage for this (or any other) data, you'll probably be fine. Of course you'd be in the same boat if you pulled the plug on the nfs server during/after rsync-over-ssh ran (e.g. rsync returns having 'finished', nfs server crashes, uncommitted data in the write cache is now lost leaving inconsistent data on disk).

Although not an issue with your test (rsyncing new data), do be aware that rsync over ssh can make significant CPU and IO demands on remote server before a single byte is transfered while it calculating checksums and generating the list of files that need to be updated.