Cisco – Configuring 4 Cisco 2960 Switches together with a pair of Force10 S55 Core Switches

Presumably your 4 Linux servers have at least two nics? Do they actually need the combined bandwidth of both nics to handle their normal load? If not then some of the bonding modes provide for redundancy without using etherchannel.

Anway, see the bonding document it provides lots of several example HA situations depending on what you need, and what your network hardware is capable of.

Excerpt from bonding.txt

11.2 High Availability in a Multiple Switch Topology
----------------------------------------------------
...

    Below is a sample network, configured to maximize the
availability of the network:

                |                                     |
                |port3                           port3|
          +-----+----+                          +-----+----+
          |          |port2       ISL      port2|          |
          | switch A +--------------------------+ switch B |
          |          |                          |          |
          +-----+----+                          +-----++---+
                |port1                           port1|
                |             +-------+               |
                +-------------+ host1 +---------------+
                         eth0 +-------+ eth1

    In this configuration, there is a link between the two
switches (ISL, or inter switch link), and multiple ports connecting to
the outside world ("port3" on each switch).  There is no technical
reason that this could not be extended to a third switch.

11.2.1 HA Bonding Mode Selection for Multiple Switch Topology
-------------------------------------------------------------

    In a topology such as the example above, the active-backup and
broadcast modes are the only useful bonding modes when optimizing for
availability; the other modes require all links to terminate on the
same peer for them to behave rationally.

active-backup: This is generally the preferred mode, particularly if
    the switches have an ISL and play together well.  If the
    network configuration is such that one switch is specifically
    a backup switch (e.g., has lower capacity, higher cost, etc),
    then the primary option can be used to insure that the
    preferred link is always used when it is available.

broadcast: This mode is really a special purpose mode, and is suitable
    only for very specific needs.  For example, if the two
    switches are not connected (no ISL), and the networks beyond
    them are totally independent.  In this case, if it is
    necessary for some specific one-way traffic to reach both
    independent networks, then the broadcast mode may be suitable.

Given the precise hardware you mention I do not have anything to add that has not already been mentioned.

The idea of having two EtherChannels, one to each switch in the rack, per ASA could be simplified quite a bit by using stackable switches. I realize that if you already have the hardware this is moot - however it could simplify future implementations with a slight change to hardware acquisition.

The Cisco 2960S line (the latest model refresh in the Catalyst 2960 series) support stacking through the FlexStack stack module -- similar to how the 3750's are stackable with StackWise[+]. FlexStack and StackWise are not the same, but from an administrative standpoint they yield many of the same results. For those that don't want to plunge into chassis switches, Cisco's stacking capabilities on the 2960S's and 3750[V2,E,X]'s provide a handful of similar functions.

In this case specifically cross-stack EtherChannel can yield much simplification. With cross-stack EtherChannel it would be possible to configure a single EtherChannel from one ASA with one PHY interface going to the first switch in the rack and a second PHY interface in the EtherChannel going to the second switch in the same rack. Additionally, the switches in the stack (in the same rack) do not need EtherChannels configured between them -- as their inter-connectivity is provided via 2960S FlexStack.

FlexStack cables can be up to 3m in length -- depending on how close your racks are together you may be able to stack all four switches.

With multiple ASA's in HA the desired redundancies can be achieved with quite a bit simplified.

I bring this up because I have similar objectives you list met with ASA's in HA and stacked 3750X's at a few client data center sites.