I'm trying to transfer a set of large files internationally using SFTP, but I am finding my international partner can't get upload speeds above ~50k despite very good connections on either side. We can get multiple connections uploading at this speed (so not bandwidth?), but no single upload improves in speed, which is a problem as many files are several gb in size.
The SFTP is being hosted using the standard Apple OSX "Remote Login" SFTP system.
Is there a way to improve upload speeds, or is there a different SFTP host that would help? It's not clear to me if this is a configuration problem or an inherent limitation of the protocol.
(For security reasons I need to be using an end-to-end encrypted peer-to-peer connection — no cloud services).
Best Answer
With OpenSSH
sftp
client (which you seem to use), you can use:-R
switch to increase request queue length (default is 64)-B
switch to increase read/write request size (default is 32 KB)For a start, try to double both:
It probably does not matter much, which of them you increase.
Increasing either should help to saturate your high-latency connection. With the above settings, it will keep 8 MB worth of data flowing in the pipe at any time (128*64K=8M).
Note that this helps with big file transfers only. It won't have any effect, when transferring a lot of small files.
For some background and a discussion about other (GUI) SFTP clients, see the "Network delay/latency" section of my answer to Why is FileZilla SFTP file transfer max capped at 1.3MiB/sec instead of saturating available bandwidth? rsync and WinSCP are even slower.