Ssh – AWS Network ACLs Breaking SSH Connectivity

It's a security boundary to have a private subnet that you can control with different security groups from the public subnet. If one of your instances in the public subnet were hacked, it will be that much more difficult to hack into instances in the private subnet if you are not too liberal in your access policies.

For posterity, this is a 5 step process. Bastion host refers to the public facing server, which you'll want to harden against potential attacks and through which your connections will travel to servers within your private subnet.

Step 1: Enable IP Forwarding (Bastion Host)

SSH to the bastion host and at the prompt, execute the following command:

echo 1 > /proc/sys/net/ipv4/ip_forward

Step 2: Modify IP Tables (Bastion Host)

SSH to the bastion host and at the prompt, execute the following commands:

sudo iptables -t nat -A PREROUTING -p tcp -m tcp --dport {PUBLIC_SSH_PORT} -j DNAT --to-destination {PRIVATE_IP_ADDRESS}:22
sudo iptables -t nat -A POSTROUTING -p tcp -m tcp --dport {PUBLIC_SSH_PORT} -j MASQUERADE

where:

1. {PUBLIC_SSH_PORT} is the port on the bastion host you plan to connect through
2. {PRIVATE_IP_ADDRESS} is the address of the server in the private subnet you plan to connect to

For more detailed information on what these configurations mean, you can run the following commands at the command prompt to view the man pages:

man iptables
man iptables-extensions

Step 3: Configure Security Groups (Amazon AWS Console)

Login to your Amazon AWS console and navigate to the Security Groups dashboard. Edit the security group assigned to your bastion host and add:

1. Custom TCP Rule to allow inbound and outbound connections on {PUBLIC_SSH_PORT} from whatever IP address you will be connecting from
2. Custom TCP Rule to allow inbound and outbound connections for All Traffic to/from the security group of the private server (type in the security group ID in the Custom IP field) - you could restrict this to just SSH traffic but if you have other services and servers running that you need to access, this is the simplest configuration

Edit the security group assigned to your private server and add:

1. Custom TCP Rule to allow inbound and outbound connections for All Traffic to/from the security group of the bastion host (type in the security group ID in the Custom IP field) - you could restrict this to just SSH traffic but if you have other services and servers running that you need to access, this is the simplest configuration

Step 4: Disable Source/Destination check (Amazon AWS Console)

Login to your Amazon AWS console and navigate to the Instances dashboard. Click on the bastion host that you just configured to forward connections. Disable source/destination checks from the action menu. This is the only device for which source/destination checks need to be disabled.

Step 5: Make Configurations Persistent (Across Server Restarts)

Modify the /etc/sysctl.conf file, and update the following line (change 0 to 1):

# Controls IP packet forwarding
net.ipv4.ip_forward = 1

Save the iptables configuration (to /etc/sysconfig/iptables) so it will be reloaded on boot:

sudo service iptables save

And finally, make sure that iptables itself starts up on boot:

sudo chkconfig iptables on

NOTE: You don't need to setup and configure a separate Amazon NAT instance to gain SSH access to private servers. The NAT instance is ideal if you plan to manage Internet access for various devices you may have within your private subnet.