I have a new server in a private subnet in an AWS VPC. I have a NAT instance in the public subnet of my VPC, and can connect out to remote servers fine. However, when I try to scp files, things seem to hang.
ryan@sever-in-vpc:~ $ scp -vvvv myfile www1.domain.com:
...
debug2: exec request accepted on channel 0
Sending file modes: C0664 42625 myfile
debug2: channel 0: rcvd ext data 25
myfile 100% 42KB 41.6KB/s 00:00
Sink: C0664 42625 myfile
debug2: channel 0: written 25 to efd 6
debug1: client_input_channel_req: channel 0 rtype keepalive@openssh.com reply 1
debug1: client_input_channel_req: channel 0 rtype keepalive@openssh.com reply 1
The "…" includes host key checking, authentication succeeding, I can give more if necessary. On the remote side, I now have "myfile" in my home directory on the remote server with a size of 0 bytes. The "debug1: client_input_channel_req: channel 0 rtype keepalive@openssh.com reply 1" message repeats seemingly until I kill the scp command. (I've left it running as long as five minutes… "myfile" is only 42625 bytes.)
It seems to me like the sending side thinks it's sent all the bytes, but the receiving side hasn't written them to disk.
Looks similar to the problem this guy was having, but no solution for him either. Any thoughts for things I can look into?
Best Answer
Turns out, this was related to the way I had VPC set up in AWS.
MTU auto-discovery should work, but depends on return ICMP traffic. It turns out that though we were allowing ICMP into our private subnet via AWS Security Groups, we were not allowing ICMP in our VPC network ACL. Opened that up and the MTU-based problem went away.
So be sure to check your VPC ACLs in addition to your Security Groups.