Would anyone know how to associate multiple Elastic IPs to a single instance via the Amazon SDK ? In Ruby, I've tried to use both the aws-sdk and fog gems, which work fine for a single address, but error out trying to assign multiple.
Via the Web UI, this would be done by adding additional private ips and then assigning the public ip to the network interface + private ip however I'm not any private ip parameters in SDK.
Best Answer
Elastic IPs working
By default, instance in the VPC will only get an IP from a private subnet within the VPC, usually somewhere in the 10.0.0.0/8 subnet. This means that while those instances can access other instances within that same subnet, they won’t be able to connect to the wider internet.
One way to give every machine access is to setup a NAT machine which all machines use to route traffic through. We won’t go any deeper into this option.
The alternative is to give every machine it’s own world-routable Elastic IP address. Amazon will route traffic to the elastic ip to your internal IP. This makes this setup simpler from a network topology perspective.
Routing in the VPC works through routing tables which are set up for each subnet. You can view your routing table in the VPC tab of the AWS console. By default, requests within your subnet are routed locally, and requests outside of that range are routed through an ‘internet gateway’.
For example, let’s say that you have an instance with IP 10.0.0.1. It has an Elastic IP associated with it, let’s say 1.2.3.4. Now whenever you access other internal machines on the 10.0.0.0/8 subnet, your request will originate from your private IP and nothing will happen to the traffic.
When you access a machine outside of your subnet, the packets are routed through the internet gateway, a VPC device with a name like igw-9d7534f2. The gateway (which isn’t an actual EC2 instance, just an opaque AWS system) accepts the request and then looks up the internal IP from which the request originates, and checks if this IP is associated with any Elastic IP’s. If so, the request is rewritten to appear as if the origin is the elastic IP, and then send out over the internet. When a package returns to the gateway with an elastic IP as destination, the gateway checks if the elastic IP is associated with an internal IP, and if so rewrites the package to that IP. The package is then forwarded into the private subnet.
This is how the one-to-one NAT of elastic IP’s works. The advantage of this approach is that machines within the subnet need to have no knowledge of their elastic IP’s. This makes it easy to switch elastic IP’s of a running instance, since the only thing that needs to be updated is the mapping table on the internet gateway. The external IP can change many times; since the packages are rewritten to use internal IP’s before they arrive at the instance, the instance will never know this.
Multiple Network Interfaces
In the past, Amazon added an option to add multiple network interfaces to a VPC instance. These interfaces appear as separate ethernet cards on your machine, and will have a separate internal IP. Separate interfaces also means you need to set up your routing between these interfaces correctly: if you send out a packet over the wrong interface, the packet will simply be dropped.
These network interfaces were introduced to give you the option of connecting different subnets together: you can have one interface in each subnet, and then route trafic between them as you like. Since each subnet has its own IP range, it is easy to route traffic to the right network interface.
For each internal IP, it is possible to associate an external Elastic IP. While this is possible, you get into a bit of a tricky routing situation now. Both internal IP’s (on eth0 and eth1) have an associated external IP, and should be able to make requests to the wider internet, by having the internet gateway translate their internal IP to the Elastic IP. However, in order to do this correctly, the requests have to be send out over the correct interface. Amazon itself admits that this is not simple:
By default, Linux has a routing table which routes all traffic outside of your local subnet to a single interface. You can look this up with route:
Here, all internet-wide traffic will go out over interface eth0. Even packets with a source ip of the eth1 interface will go out over eth0, and then will be silently discarded instead of being NAT’d to the right elastic IP. While it’s possible to do source-based routing to fix this, this isn’t trivial to set up (note that you need to run dhclient after adding another interface to a machine):
Multiple IP Addresses
This way we can avoid some of the routing nasties from above: all packets will be sent over eth0, regardless of which IP you have. Because of the one-to-one mapping of public and private addresses, you first need to add some new private addresses to one of your instances.
Amazon’s API has been updated to support assigning secondary private IP’s, but for this example it’s easier to just go to the AWS Console. In the EC2 tab, go to Instances. Find the instance you want to update, right-click and choose “Manage Private IP Adresses”.
You’ll now see that the instance has two private IP’s in your subnet (since an interface is locked to specific subnet, all IP’s on that interface have to fall within that same subnet). Nice! However, if you now check your instance:
It won’t have the new IP yet. Let’s try retrieving it over DHCP!
Nope, we only get our primary private ip! With DHCP you can only get a single address. In the future, perhaps Amazon will add support for multiple IP’s using some client identifier or another mechanism, but for now you’ll have to add the address manually. Hopefully Ubuntu’s cloud-init will add support for this in the future, so we don’t have to do it ourselves.
Amazon suggests creating a new virtual interface and then bringing that up. While that will work, it means you store your interface state on your hard drive, and any changes you make on AWS you will have to propagate to these files.
For testing, we can do something simpler:
This will check the EC2 instance meta-data for all internal IPs associated with eth0, and adds all the secondary IP’s.
You can now asociate another elastic ip to the secondary IP in the AWS Console:
And now you have multiple external IP’s!
While this works for testing, it’s not persistent across reboot. You can create an upstart or cloud-init script to do that for you. Even then, ip addresses aren’t automatically added when you add them through EC2. I’m not sure if there’s a good way to do that. Finally, this script also won’t remove any local addresses you may have removed in the meantime.
Hope this will help you.