I am new to this whole topic and I try for days now to figure out how to assign multiple public ip-addresses to KVM-guests through a KVM host. I found tons of examples howto get such a setup with 1 public IP running.
Here is my setup:
The Server has only one NIC/MAC and runs 2 KVM-Guests with apache(and other stuff). Both guest-environments are ubuntu server 11.10 and must run in separate VMs. The 5 public ip-addresses are used to handle SSL-certificates and other stuff. The first VM should use 3 of the 5 addresses/certificates. The second VM gets the rest. The apache-stuff is configured correctly.
I have tried a number of differend ways via iptables to route the traffic from the hosts NIC to the guest-NICs. In spite of the fact that one way was the right one but only wrong implemented, I leave the details untold to leave you unprepossessed. The question is: Whats the ideal way it should be done?
The following conditions should be met:
- Apache must get the original ip-address of the visitor
- Apache must know, what public-ip address was ultilized to use the right ssl-vhost
- The traffic must not be routed through a (reverse-)proxy on the host, since there are 2 other non-http-services, on other VM-guests, that should be accessible from public. And: Only sshd should listen directly on the host – nothing else
- Each VM should be able to access the internet directly.
- The network in the data-center is switched MAC-based. As I figured out, the only way to comunicate with the internet is through eth0 and its MAC-address.
If I would discard all the virtualization-stuff, this would be perfectly easy, as apache get the request directly from a specific ip-address.
I am open for any working solution.
Best Answer
Use a bridge on your dom0 (e.g. KVM Host) WAN interface. This requires installing
bridge-utils
package. Since this is Debian-based distro, you may configure it in/etc/network/interfaces
:Pre-up commands set up TAP interface to connect your KVM guest to a bridge. Note that this setup allows to run kvm from non-privileged user guest1. Note that setting
net.ipv4.ip_forward = 1
with sysctl might be usefull as well.I have used
ip tuntap
command fromiproute2
package. It's not yet documented in the Debian package but soon will be available in upstream's manual page. Since this package is installed on every Debian-based server, you won't need to installuml-utilities
oropenvpn
package to just create these interfaces.This approach sure lacks some elegance to manage lots of tap interfaces, because you'll need to create similar pre-up and post-down lines as for
tap_guest1
interface. This can be fixed by writing additional scripts in/etc/network/pre-up.d
and/etc/network/post-down.d
. It is also a problem if you want to reconfigurebr_wan
interface with ifdown/ifup scripts while KVM guests are still running — you'll need either to remove all interfaces excepteth0
from bridge configuration and detach them from bridge manually (don't forget to attach them back after bridge reconfiguration then) or shutdown all KVM instances running on a bridge.Another way, perhaps more clean, is to write custom ifup script for KVM itself and use it in
script
option for your NIC. You can get an example in/etc/qemu-ifup
. See kvm manual page for details.Then you can run your KVM box like this:
Setting several IP addresses on one interface for your KVM guest can be done manually with command
Or permanently in
/etc/network/interfaces
like this:Note that if your datacenter/provider does not expect you to reveal additional boxes on the same net he might not configure them and they will be unavailable. In this case you might want to create internal bridge and use iptables to forward packets between your WAN interface and this bridge using DNAT and SNAT. Assuming your local virtual bridge network is 10.0.0.0/8, your guest1 is 10.0.0.2 you'll need this:
Note that you'll need as much DNAT commands as external IPs per KVM guest you have, but only one SNAT rule to give access to the internet. Also you can allow only HTTP/HTTPS/SSH traffic by allowing only desired ports. If you omit the
--dport
statement then all ports will be forwarded. Your KVM guest should have static network settings with KVM host as default gateway unless you're willing to host DHCP server.