I have done load balancing using both lartc.org and iptables methods, and I find that the iptables method is easier to understand and implement. The only downside is that you need a fairly recent iptables version to be able to use statistic module
Let's suppose a few things:
LAN: eth0: 192.168.0.1/24
ISP1: eth1: 192.168.1.1/24, gateway: 192.168.1.2/24
ISP2: eth2: 192.168.2.1/24, gateway: 192.168.2.2/24
So here is how I would do by using iptables method:
Route tables
First edit the /etc/iproute2/rt_tables to add a map between route table numbers and ISP names
...
10 ISP1
20 ISP2
...
So table 10 and 20 is for ISP1 and ISP2, respectively. I need to populate these tables with routes from main table with this code snippet (which I have taken from hxxp://linux-ip.net/html/adv-multi-internet.html)
ip route show table main | grep -Ev '^default' \
| while read ROUTE ; do
ip route add table ISP1 $ROUTE
done
And add default gateway to ISP1 through that ISP1's gateway:
ip route add default via 192.168.1.2 table ISP1
Do the same for ISP2
So now I have 2 route tables, 1 for each ISP.
Iptables
OK now I use iptables to evenly distribute packets to each route tables. More info on how this work can be found here (http://www.diegolima.org/wordpress/?p=36) and here (http://home.regit.org/?page_id=7)
# iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
# iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j ACCEPT
# iptables -t mangle -A PREROUTING -j MARK --set-mark 10
# iptables -t mangle -A PREROUTING -m statistic --mode random --probability 0.5 -j MARK --set-mark 20
# iptables -t mangle -A PREROUTING -j CONNMARK --save-mark
NAT
Well NAT is easy:
# iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
# iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE
For using multiple outgoing connections, all you need is something to route the traffic appropriately. There are a number of free linux/BSD router distros that will work fine, such as m0n0wall.
However, to share the same set of public IPs on multiple incoming connections, ISP participation is required.
This is indeed referred to as multi-homing, and requires the use of BGP, an assigned AS number, and ownership of an IP space.
This is not often feasible for such small IP spaces; you may be able to get a redundant link from one ISP, in which case they will take care of the multi-homing, but that doesn't protect you against that ISP going down...
An alternative is to host your critical services in a datacenter, which will be redundant in all respects (power, cooling, hardware, connectivity) - but you will have to compare costs.
Best Answer
The answer to the specific question about to set up the existing network architecture to support load balancing between the two existing firewalls is to set up a load balancing router behind the firewalls and in front of your LAN, or two routers in HA if you need failover from hardware failure.
This is achievable with a Cisco router that support IP SLAs. Eg. we have done this with the Cisco 800 series before. Using multiple gateways, the router can route out both connections (achieving the load balancing requirement) and if required you can used policy based routing to send all traffic via a specific link based on source or destination IP.
The router can be set up to monitor two different IP addresses, one for each ISP, and set up to route traffic for those IPs only out their respective links. If one of those IPs is not reachable the IP SLA can be configured to remove the route through that ISP, hence routing only through the other ISP that is still available (satisfying the failover requirement). Once the failed ISP comes back online the router can be configured to automatically add the route back in and the links are load balancing again. This is a relatively complex setup and a sample config depends on various factors, including the version of IOS, the types of links, latency, reliability of links, network topology, incoming traffic requirements etc.
This setup also requires quite a bit of testing of the failover and failback logic in the case of an ISP failure. If the failover between the ISP links is too sensitive you'll end up with flapping routes, and if not sensitive enough it will take a long time to fail over and there will be intermittent traffic disruptions in both cases. Note that this method does not use any fancy routing protocols, it's set up with "roll-your-own" logic.
Deviating from the specific question being asked, the best option is likely to be to decommission one or both of the existing firewalls and replace with an HA firewall solution that supports outbound load balancing and failover. It's a simpler solution, and different firewall technologies are typically used in-line rather then in parallel, with the theory being that dual layer multi-vendor firewalls give an additional layer of security. There are many firewall vendors and technologies that support outbound load balancing (eg. PFSense, F5, many more), and determining the best one would be best done through further investigation.
You can read about Cisco IP SLAs here, and about Cisco Policy Based Routing here.