We are currently trying to rearrange our network for performance. Having already begun to solve the issue that our network diagram looked like a straight line, I am now looking at the 4 nics for the virtual host.
As they currently are, 3 of the nics go to one switch, and one to another – but I get the feeling this was not so much a choice for design so much as a choice for what ports were free when it was installed.
Given an extremely simplified version of what our network looks like without the virtual host:
I am not sure whether I should be hooking all 4 of the nics to the central switch like so:
or distributing them so that any client has a short route to the host (but only to one of it's NICs), like so:
for the record, we are using vSphere eSXI vSphere 5.1, and the virtual machines being run provide entirely in-network services – none of them are internet-facing.
The layout of the nics <-> the vSwitch environment is:
Best Answer
There are reasons that you'd want to spread your uplinks from a vSwitch out between multiple physical switches -- but mostly for redundancy purposes, not for decreasing the length of the path that the traffic will need to take for communication between a client and a virtual machine.
How exactly this behaves depends on the configuration of your vSwitch -- in the NIC teaming tab, the settings that dictate how the NICs will be used are the Load Balancing drop-down, as well as the active/standby state of the adapters.
The Port ID and MAC hash configurations will both effectively assign each VM's traffic to a specific physical NIC - so traffic outbound from that VM (and inbound to the VM, due to the switches' MAC address table) will be stuck to the same physical NIC until something changes (like the NIC topology changing due to one going down, for instance).
The other load balancing option is IP hash, but you don't want that when uplinking to multiple switches; you'll get all kinds of MAC address flapping.
In general, layer 2 switching is not capable of taking the shortest physical path to something; there's just one working path, at least until TRILL starts taking off.
You can potentially manually control which switch a specific VM tends to be pointed at; would that be helpful?