What I would like to achieve
I want to securely spread an existing internal subnet over multiple buildings. That means that I have two locations with virtual machines that need to be within the same subnet. The idea is that the virtual machines (having a static IP) can be migrated from one location to the other.
The (physical) host machines are connected to a switch at each location. So, if there wasn't any security or cost problem I would simply connect both switches with a network cable:
[Machines]—[Switch A] <—- LONG CABLE —> [Switch B]—[Machines]
What I would like, is to replace this long cable by an encrypted tunnel using two gateways that don't need to care about IP adresses or routing and just take any incoming packets encrypt them and send them to the other gateway via an encrypted tunnel. The other gateway then decrypts the packets and sends them to the remote switch. This would physically look like this:
[Machines]—[Switch A]—[GATEWAY A] <– INTERNET –> [GATEWAY B]–[Switch B]—[Machines]
I would like to avoid that the gateways need any IP adresses within the subnet. The rules shall be completly port-based:
- Incoming data at port 1: route through tunnel interface
- Incoming data at tunnel interface: route via port 1
The two Gateways would have a static, routeable IP address to establish the tunnel. The encryption shall be strong (at least AES128, SHA256, DH2048; shared secret is fine), which simple PPP type tunnels don't support. So an additional/seperate encryption layer might be needed.
I've only MikroTik Routers available. So I would prefer to use them. However, I'm mostly looking for the 'magic words' (protocol names and the like) and the right combination of technology that allows me to do that. So, if you know how to do it with Cisco routers or HP routers, it would probably also help, if you explained how you would just do it with that ones…
Questions/Attempts
What kind of firewall filters and protocols can I use to achieve this?
My first idea was to use IPsec to span the encrypted tunnel. But, then I would need to define IPsec Policy that is physical-port based. But there is only an option to define that data from/to a special IP-adress / IP-port combination.
So IPsec would just work as encryption layer for another tunnel type (PPTP, SSTP, L2TP and OVPN are currently supported by the MikroTik RouterOS). As PPP-Tunnels typically don't support strong encryption, I would let IPsec do this job and span the unencrypted-PPP-Tunnel through the encrypted IPsec-tunnel.
Ok, now we had at least some tunnel interface, that we can use like a outgoing port. However, I'm kind of lost here. I don't find that possibility to say: "a frame with incoming at has to be sent out via interface " and "a frame incoming at has to be sent out via interface ".
I'm not often working at Layer2… so I'm actually looking for the right 'term' or 'category'. I could imagine finding it at the IP-Firewall (mangle->prerouting) or something like that, but I assume that's already Layer 3 stuff…
Do I just need to setup a bridge? If so, how can I add the tunnel-interface to the bridge (preferably using the winbox-Interface)? Does the bridge need a MAC adress?
Just in case this attempt is a dead end:
I also found "EoIP", "IP Tunnel" and "GRE Tunnel" at the "Interface" setting. But I've no real idea what they can do… So just in case, let me know which of them is worth investigating…
Also, If there is a more easy-and-clean solution don't mind just telling me your solution… You don't have to continue my above attempts, if there is just an easier way!
Best Answer
You can accomplish what you ask by using a combination of bridges, EoIP tunnels and IPsec.
First you create an EoIP tunnel on both sides so that both routers can talk to each other.
EoIP is a MikroTik proprietary tunnel that works quite like GRE proto 47.
It can forward whole ethernet frames not just IP (like IPIP or TAP interface for example) making it ideal to 'expand' L2 broadcast domains over L3 links.
Then you setup IPsec to encrypt the communications over the tunnel.
On the latest versions of Mikrotik this can be done automatically from the EoIP interface settings, though I prefer setting this up manually for better control.
You apply the IPsec policies on the EoIP tunnels endpoint IPs (so encrypting the whole EoIP tunnel traffic) and not for the networks passing through the tunnel.
And finally you create a bridge on both routers and attach the ethernet port(s) and the EoIP tunnel on each side.
This way you can expand the L2 broadcast domain to the remote side and the bridge will take care of which packets should go over the tunnel and which should stay locally.
Bridges are like software switches so they keep a table of which MACs are on which ports and automatically forward packets that are destined to which ever port. All broadcast traffic will be forwarded to the tunnel of course. More info on bridges
On the official MikroTik documentation this scenario is already documented (with the exception of the IPsec encryption).
And you also need to setup IPSec. More info here
Keep in mind that IPsec while the most secure way to communicate on Mikrotik at this moment it's rather CPU hungry unless your routers have hardware acceleration. So if you are using simple routerboard don't expect anything more than 10-15mbits of throughput.
If your routers have dynamic internet IPs then you will need to use the IP > Cloud feature of MikroTik to get a Dynamic DNS and then use those DynDNS with a little scripting to automatically update the IPs on the EoIP tunnels and IPsec peers and policies when your public IPs change.
Finally I assume that your connections to the internet have at best a MTU of 1500bytes. Using the tunnel and EoIP you will need to lower the EoIP tunnel interface MTU in the realm of 1400bytes (you'd better do some testing to find out the best value for your setup/traffic).
For my setup/traffic I use 1400bytes. Unfortunately due to the nature of IPsec depending on the size of your packets the overhead is variable so you can't always get the best out of it.