Linux – Faster file writing on sshfs mounted directory

linuxmountsshfs

Here is the following scenario: I have server1 and server2 and using sshfs I mount a directory from server2 on server1, let's say /mnt.
Now, if I try to unpack a 50K files archive on server1:/mnt it takes 13 mins to complete while doing the same on server1:/local_dir will only take some 5 seconds.

I was wondering if there is any way/configuration that could speed up the things.

Best Answer

It's slower since it has to unpack them to a remote server. The speed difference is due to network delay, sshfs overhead, etc. For every file, you have to send the data to server2, wait for response, and then do it again for the next.

If you want to speed it up a bit*, upload the package to server1:/mnt and unpack it on server2.

*it will still be slower then on server1:/local_dir ofcourse

Related Solutions

Linux – Why doesn’t SSHFS let me look into a mounted directory

I've found the answer myself. The problem was that I didn't use the option allow_other.

sshfs -o allow_other -o kernel_cache -o auto_cache -o reconnect \
  -o compression=no -o cache_timeout=600 -o ServerAliveInterval=15 \
  xxx@yyy.yyy.yyy.yyy:/mnt/content /home/xxx/path_to/content

To use this option you have to set the option user_allow_other in /etc/fuse.conf. When I did this I had another problem. The file /etc/fuse.conf haven't had read permissions for other users on my Ubuntu box. So I've changed that too and now I can access the directory with any user.

Linux – Is this a Linux NFS client bufferbloat

OK, here's my answer.

Related to https://bugzilla.redhat.com/show_bug.cgi?id=688232 with kernel 2.6.18 and 2.6.32 as shipped with RedHat (I haven't time to re-validate this with vanilla newer kernels), on a NFS client (v3 / tcp / default mount options), when one is writing to a file, the kernel also needs to update the timestamps of this file. While the file is being written, if another process wants the metadata of this file (such as when doing a stat on this file or ls -l in its parent directory), this reader process gets delayed by the kernel until the write is finished.

At the NFS level, I can see that the kernel will issue the GETATTR call only after all (I am not sure on this, but in my tests up to 5GiB, the stat time seemed to match the dd time) the WRITE. The bigger the write is, the longer is the wait.

With a slow NFS server or a server with a lot of RAM, that delay can be minutes. When the stat(2) gets put to sleep, one can monitor /proc/meminfo for NFS_Unstable or Writeback which shows how much data is flight.

I am not sure why the kernel does this, but at least now I understand the behaviour. So there's no bufferbloat, but some operations are serialized.

Best Answer

Related Solutions

Linux – Why doesn’t SSHFS let me look into a mounted directory

Linux – Is this a Linux NFS client bufferbloat

Related Topic