Ssh – Choosing a buffer size for large file transfer – SSH

buffermemoryssh

I am trying to transfer large files over an internal network using Secure Shell SSH Client (3.2.9). The default buffer size is 32KB, which should be plenty, but my specific case makes me wonder if I could get a faster/more reliable transfer by increasing it:

  1. The transfer is one-way. From a server running Unix, directly to a 1TB HDD attached to the desktop via eSATA. The local OS is Windows 7.
  2. I am transferring 24x 15GB text files for a DNA experiment.
  3. The current speed is averaging 12MB/s (buffer size=32KB)

Basically I've let the transfer run a couple hours and keep getting a fatal error "Out of Memory," even though I'm not even using half my 8GB RAM. Could upping the buffer size help?

Best Answer

Something is probably wrong with your SSH client. Try doing the same with putty's pscp.

More reliable.. It doesn't get more reliable than TCP :) Regarding speed, SSH shouldn't be a bottleneck. 12MB/sec sounds like 100Mbps, so unless you're on a 1Gbps network all the way it sounds about right.

Related Topic