We have seen almost the same issue. We are using NLB with multicast (but not IGMP) to load balance 14 web servers across two ESX 4 servers plugged in a pair of stacked Dell PowerConnect6248. The nlb was working but the performance was terrible. We tried chnaging everying on nlb (unicast, multicast, igmp) and vmware switch (promicous, nitify switch, etc) and could not make it work. We added multicast MAC to dell bridge and arp tables all to no effect. We eventually solved it by turning routing off the vlan on the PowerConnect (i.e. using simple layer 2 VLAN) and using an external router to route traffic.
Would love to know how to using the routing on Dell to make this work as it should be supported.
If possible, go to dynamic routing. Given the right physical setup, using static routes with different metrics will give you decent fail-over behaviour.
A router normally doesn't track reachability for next hops, it tracks "interface up" or "interface down" and if at least one interface that is a suitable egress for a given next hop is "up", the next hop is considered reachable. At that point, the only thing that would cause packets not to be sent to a next hop, up or down, is that there's no ARP resolution and as ARP requests are usually cached for quite a while (Cisco default is, I believe, 4 hours), you may be in for a long wait.
In "Cisco-land", using static routes with different (administrative) distances is called "floating statics" and are usually used for fail-over from a serial link to another link, as the serial link is (usually) point-to-point (may not be the case, if you're using FR or other serial link protocols able to provide multi-point) and has enough signaling to be able to flag "other end is unreachable (unlike, say, most "metro Ethernet", where there's usually multiple L2 hops between the two L3 end-points, so a break somewhere in the transmission path is usually not visible as a downed interface).
So, in short, if you can arrange that your 5510 is connected on a dedicated switch-port, with a /30 network, on a single core switch and the 5510 will NOT cause the switch-port on the core switch to signal as up when the 5510 is switched off (or you're willing to take the time-hit of someone to have to either modify the routing or unplug the cable), floating statics may be exactly all you need. Well worth investigating, but I'd probably look into configuring dynamic routing, at least for the VPN routes.
Best Answer
According to the CLI manual, the feature is not available.