Linux: Using bonding to connect two servers for faster file transfer

My setup is two Dell R720 servers which are connected over 4 gigbit ports each to a Cisco WS-C2960S-24TS-L switch which in turn is connected to the Internet via 100MBit.

The servers are running Wheezy with an openvz (redhad) kernel:
2.6.32-openvz-042stab081.3-amd64

What I want is faster file-transfer between the two servers and some level of fault tolerance.

I managed to setup bonding and tried bonding modes balance-rr, 802.3ad and balance-alb. All worked in terms of me being able to connect to the servers. But I don't get any speedup in datatransfer between them.

(DELETED: I do understand that balance-rr only works with xover cabeling.)

Looking at the traffic count of ifconfig and the individual interfaces I see:

• 802.3ad: outgoing traffic using only the first interface. This is even true if transfering to another host with different mac-address.
• balance-alb: outgoing traffic "somehow" unevenly distributed between interfaces but incoming traffic only on one interface

The Kernel docs tell me, that balance-rr mode needs: The balance-rr, balance-xor and broadcast modes generally require that the switch have the appropriate ports grouped together.
The nomenclature for such a group differs between switches, it may be
called an "etherchannel"

So the question is:

• What is the right mode for me to use and how do I set it up so that it works?

• If this is not possible generally it would help somehow having a setup which would use different interfaces for server/server and server/internet connections. But this has to use bonding and not different internal/external ip-adresses. (Because this in turn would make the openvz setup unnecessarily difficult)

Thanks in advance!

UPDATE: Having played with the switch I have set up two etherchannels for the two servers in "active" mode (is this correct?). But using 802.3ad as bonding method on the linux side I didn't see any changes in behaviour/speed.

UPDATE2: Sorry. Seems like now outgoing traffic uses different interfaces. Probably depending on the destinations mac address. Is this the best I can do?

UPDATE3: Only to show what I'm talking about:

root@warp2:/ssd/test# iperf -c 10.0.0.1
------------------------------------------------------------
Client connecting to 10.0.0.1, TCP port 5001
TCP window size: 23.8 KByte (default)
------------------------------------------------------------
[  3] local 10.0.0.2 port 55759 connected with 10.0.0.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  2.16 GBytes  1.85 Gbits/sec

root@warp2:/ssd/test# iperf -c x.x.x.x
------------------------------------------------------------
Client connecting to warp1, TCP port 5001
TCP window size: 23.8 KByte (default)
------------------------------------------------------------
[  3] local 80.190.169.17 port 54861 connected with x.x.x.x port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.10 GBytes   944 Mbits/sec

First test is with 2 nics using balance-rr and currently in two vlans (one for every nic pair simulating x-link cables.)

Second test is with 2 nics using 802.3ad and EtherChannel.