Virtual LANs (VLANs) are an abstraction to permit a single physical network to emulate the functionality of multiple parallel physical networks. This is handy because there may be situations where you need the functionality of multiple parallel physical networks but you'd rather not spend the money on buying parallel hardware. I'll be speaking about Ethernet VLANs in this answer (even though other networking technologies can support VLANs) and I won't be diving deeply into every nuance.
A Contrived Example and a Problem
As a purely contrived example scenario, imagine you own an office building that you lease to tenants. As a benefit of the lease, each tenant will get live Ethernet jacks in each room of the office. You buy a Ethernet switch for each floor, wire them up to jacks in each office on that floor, and wire all the switches together.
Initially, you lease space to two different tenants-- one on the floor 1 and one on 2. Each of these tenants configures their computers w/ static IPv4 addresses. Both tenants use different TCP/IP subnets and everything seems to work just fine.
Later, a new tenant rents half of floor 3 and brings up one of these new-fangled DHCP servers. Time passes and the 1st floor tenant decides to jump on the DHCP bandwagon, too. This is the point when things start to go awry. The floor 3 tenants report that some of their computers are getting "funny" IP addresses from a machine that isn't their DHCP server. Soon, the floor 1 tenants report the same thing.
DHCP is a protocol that takes advantage of the broadcast capability of Ethernet to allow client computers to obtain IP addresses dynamically. Because the tenants are all sharing the same physical Ethernet network they share the same broadcast domain. A broadcast packet sent from any computer in the network will flood out all the switch ports to every other computer. The DHCP servers on floors 1 and 3 will receive all requests for IP address leases and will, effectively, duel to see who can answer first. This is clearly not the behavior you intend your tenants to experience. This is the behavior, though, of a "flat" Ethernet network w/o any VLANs.
Worse still, a tenant on floor 2 acquires this "Wireshark" software and reports that, from time to time, they see traffic coming out of their switch that references computers and IP addresses that they've never heard of. One of their employees has even figured out that he can communicate with these other computers by changing the IP address assigned to his PC from 192.168.1.38 to 192.168.0.38! Presumably, he's just a few short steps away from performing "unauthorized pro-bono system administration services" for one of the other tenants. Not good.
Potential Solutions
You need a solution! You could just pull the plugs between the floors and that would cut off all unwanted communication! Yeah! That's the ticket...
That might work, except that you have a new tenant who will be renting half of the basement and the unoccupied half of floor 3. If there isn't a connection between the floor 3 switch and the basement switch the new tenant won't be able to get communication between their computers that will be spread around both of their floors. Pulling the plugs isn't the answer. Worse still, the new tenant is bringing yet another one of these DHCP servers!
You flirt with the idea of buying physically separate sets of Ethernet switches for each tenant, but seeing as how your building has 30 floors, any of which can be subdivided up to 4 ways, the potential rats nest of floor-to-floor cables between massive numbers of parallel Ethernet switches could be a nightmare, not to mention expensive. If only there was a way to make a single physical Ethernet network act like it was multiple physical Ethernet networks, each with its own broadcast domain.
VLANs to the Rescue
VLANs are an answer to this messy problem. VLANs permit you to subdivide an Ethernet switch into logically disparate virtual Ethernet switches. This allows a single Ethernet switch to act as though it's multiple physical Ethernet switches. In the case of your subdivided floor 3, for example, you could configure your 48 port switch such that the lower 24 ports are in a given VLAN (which we'll call VLAN 12) and the higher 24 ports are in a given VLAN (which we'll call VLAN 13). When you create the VLANs on your switch you'll have to assign them some type of VLAN name or number. The numbers I'm using here are mostly arbitrary, so don't worry about what specific numbers I choose.
Once you've divided the floor 3 switch into VLANs 12 and 13 you find that the new floor 3 tenant can plug in their DHCP server to one of the ports assigned to VLAN 13 and a PC plugged into a port assigned to VLAN 12 doesn't get an IP address from the new DHCP server. Excellent! Problem solved!
Oh, wait... how do we get that VLAN 13 data down to the basement?
VLAN Communication Between Switches
Your half-floor 3 and half-basement tenant would like to connect computers in the basement to their servers on floor 3. You could run a cable directly from one of the ports assigned to their VLAN in the floor 3 switch to the basement and life would be good, right?
In the early days of VLANs (pre-802.1Q standard) you might do just that. The entire basement switch would be, effectively, part of VLAN 13 (the VLAN you've opted to assign to the new tenant on floor 3 and the basement) because that basement switch would be "fed" by a port on floor 3 that's assigned to VLAN 13.
This solution would work until you rent the other half of the basement to your floor 1 tenant who also wants to have communication between their 1st floor and basement computers. You could split the basement switch using VLANs (into, say, VLANS 2 and 13) and run a cable from floor 1 to a port assigned to VLAN 2 in the basement, but you better judgement tells you that this could quickly become a rat's nest of cables (and is only going to get worse). Splitting switches using VLANs is good, but having to run multiple cables from other switches to ports which are members of different VLANs seems messy. Undoubtedly, if you had to divide the basement switch 4 ways between tenants who also had space on higher floors you'd use 4 ports on the basement switch just to terminate "feeder" cables from upstairs VLANs.
It should now be clear that some type of generalized method of moving traffic from multiple VLANs between switches on a single cable is needed. Just adding more cables between switches to support connections between different VLANs isn't a scalable strategy. Eventually, with enough VLANs, you'll be eating up all the ports on your switches with these inter-VLAN / inter-switch connections. What's needed is a way to carry the packets from multiple VLANs along a single connection-- a "trunk" connection between switches.
Up to this point, all the switch ports we've talked about are called "access" ports. That is, these ports are dedicated to accessing a single VLAN. The devices plugged into these ports have no special configuration themselves. These devices don't "know" that any VLANs are present. Frames the client devices send are delivered to the switch which then takes care of making sure that the frame is only sent to ports assigned as members of the VLAN assigned to the port where the frame entered the switch. If a frame enters the switch on a port assigned as a member of VLAN 12 then the switch will only send that frame out ports that are members of VLAN 12. The switch "knows" the VLAN number assigned to a port from which it receives a frame and somehow knows to only deliver this frame out ports of the same VLAN.
If there were some way for a switch to share the VLAN number associated with a given frame to other switches then the other switch could properly handle delivering that frame only to the appropriate destination ports. This is what the 802.1Q VLAN tagging protocol does. (It's worth noting that, prior to 802.1Q, some vendors made up their own standards for VLAN tagging and inter-switch trunking. For the most part these pre-standard methods have all been supplanted by 802.1Q.)
When you have two VLAN-aware switches connected to each other and you want those switches to deliver frames between each other to the proper VLAN you connect those switches using "trunk" ports. This involves changing the configuration of a port on each switch from "access" mode to "trunk" mode (in a very basic configuration).
When a port is configured in trunk mode each frame that the switch sends out that port will have a "VLAN tag" included in the frame. This "VLAN tag" wasn't part of the original frame that the client sent. Rather, this tag is added by the sending switch prior to sending the frame out the trunk port. This tag denotes the VLAN number associated with the port from which the frame originated.
The receiving switch can look at the tag to determine which VLAN the frame originated from and, based on that information, forward the frame out only ports that are assigned to the originating VLAN. Because the devices connected to "access" ports aren't aware that VLANs are being used the "tag" information must be stripped from the frame before it's sent out a port configured in access mode. This stripping of the tag information causes the entire VLAN trunking process to be hidden from client devices since the frame they receive will not bear any VLAN tag information.
Before you configure VLANs in real life I'd recommend configuring a port for trunk mode on a test switch and monitoring the traffic being sent out that port using a sniffer (like Wireshark). You can create some sample traffic from another computer, plugged into an access port, and see that the frames leaving the trunk port will, in fact, be larger than the frames being send by your test computer. You'll see the VLAN tag information in the frames in Wireshark. I find that it's worth actually seeing what happens in a sniffer. Reading up on the 802.1Q tagging standard is also a decent thing to do at this point (especially since I'm not talking about things like "native VLANs" or double-tagging).
VLAN Configuration Nightmares and the Solution
As you rent more and more space in your building the number of VLANs grows. Each time you add a new VLAN you find that you have to logon to increasingly more Ethernet switches and add that VLAN to the list. Wouldn't it be great if there were some method by which you could add that VLAN to a single configuration manifest and have it automatically populate the VLAN configuration of each switch?
Protocols like Cisco's proprietary "VLAN Trunking Protocol" (VTP) or the standards-based "Multiple VLAN Registration Protocol" (MVRP-- previously spelled GVRP) fulfill this function. In a network using these protocols a single VLAN creation or deletion entry results in protocol messages being sent to all switches in the network. That protocol message communicates the change in VLAN configuration to the rest of the switches which, in turn, modify their VLAN configurations. VTP and MVRP aren't concerned with which specific ports are configured as access ports for specific VLANs, but rather are useful in communicating the creation or deletion of VLANs to all the switches.
When you've gotten comfortable with VLANs you'll probably want to go back and read about "VLAN pruning", which is associated with protocols like VTP and MVRP. For now it's nothing to be tremendously concerned with. (The VTP article on Wikipedia has a nice diagram that explains VLAN pruning and the benefits therewith.)
When Do You Use VLANs In Real Life?
Before we go much further it's important to think about real life rather than contrived examples. In lieu of duplicating the text of another answer here I'll refer you to my answer re: when to create VLANs. It's not necessarily "beginner-level", but it's worth taking a look at now since I'm going to make reference to it briefly before moving back to a contrived example.
For the "tl;dr" crowd (who surely have all stopped reading at this point, anyway), the gist of that link above is: Create VLANs to make broadcast domains smaller or when you want to segregate traffic for some particular reason (security, policy, etc). There aren't really any other good reasons to use VLANs.
In our example we're using VLANs to limit broadcast domains (to keep protocols like DHCP working right) and, secondarily, because we want isolation between the various tenants' networks.
An Aside re: IP Subnets and VLANs
Generally speaking there is a typically a one-to-one relationship between VLANs and IP subnets as a matter of convenience, to facilitate isolation, and because of how the ARP protocol works.
As we saw at the beginning of this answer two different IP subnets can be used on the same physical Ethernet without issue. If you're using VLANs to shrink broadcast domains you won't want to share the same VLAN with two different IP subnets since you'll be combining their ARP and other broadcast traffic.
If you're using VLANs to segregate traffic for security or policy reasons then you also probably won't want to combine multiple subnets in the same VLAN since you'll be defeating the purpose of isolation.
IP uses a broadcast-based protocol, Address Resolution Protocol (ARP), to map IP addresses onto physical (Ethernet MAC) addresses. Since ARP is broadcast based, assigning different parts of the same IP subnet to different VLANs would be problematic because hosts in one VLAN wouldn't be able to receive ARP replies from hosts in the other VLAN, since broadcasts aren't forwarded between VLANs. You could solve this "problem" by using proxy-ARP but, ultimately, unless you have a really good reason to need to split an IP subnet across multiple VLANs it's better not to do so.
One Last Aside: VLANs and Security
Finally, it's worth noting that VLANs aren't a great security device. Many Ethernet switches have bugs that permit frames originating from one VLAN to be sent out ports assigned to another VLAN. Ethernet switch manufacturers have worked hard to fix these bugs, but it's doubtful that there will ever be a completely bug free implementation.
In the case of our contrived example the floor 2 employee who is moments away from providing free systems administration "services" to another tenant might be stopped from doing so by isolating his traffic into a VLAN. He might also figure out how to exploit bugs in the switch firmware, though, to allow his traffic to "leak" out onto another tenant's VLAN as well.
Metro Ethernet providers are relying, increasingly, on VLAN tagging functionality and the isolation that switches provide. It's not fair to say that there's no security offered by using VLANs. It is fair to say, though, that in situations with untrusted Internet connections or DMZ networks it's probably better to use physically separate switches to carry this "touchy" traffic rather than VLANs on switches that also carry your trusted "behind the firewall" traffic.
Bringing Layer 3 into the Picture
So far everything this answer has talked about relates to layer 2-- Ethernet frames. What happens if we start bringing layer 3 into this?
Let's go back to the contrived building example. You've embraced VLANs opted to configure each tenant's ports as members of separate VLANs. You've configured trunk ports such that each floor's switch can exchange frames tagged with the originating VLAN number to the switches on the floor above and below. One tenant can have computers spread across multiple floors but, because of your adept VLAN configuring skills, these physically distributed computers can all appear to be part of the same physical LAN.
You're so full of your IT accomplishments that you decide to start offering Internet connectivity to your tenants. You buy a fat Internet pipe and a router. You float the idea to all your tenants and two of them immediately buy-in. Luckily for you your router has three Ethernet ports. You connect one port to your fat Internet pipe, another port to a switch port assigned for access to the first tenant's VLAN, and the other to a port assigned for access to the second tenant's VLAN. You configure your router's ports with IP addresses in each tenant's network and the tenants start accessing the Internet through your service! Revenue increases and you're happy.
Soon, though, another tenant decides to get onto your Internet offering. You're out of ports on your router, though. What to do?
Fortunately you bought a router that supports configuring "virtual sub-interfaces" on its Ethernet ports. In short this functionality allows the router to receive and interpret frames tagged with originating VLAN numbers, and to have virtual (that is, non-physical) interfaces configured with IP addresses appropriate for each VLAN it will communicate with. In effect this permits you to "multiplex" a single Ethernet port on the router such that it appears to function as multiple physical Ethernet ports.
You attach your router to a trunk port on one of your switches and configure virtual sub-interfaces corresponding to each tenant's IP addressing scheme. Each virtual sub-interface is configured with the VLAN number assigned to each Customer. When a frame leaves the trunk port on the switch, bound for the router, it will carry a tag with the originating VLAN number (since it's a trunk port). The router will interpret this tag and treat the packet as though it arrived on a dedicated physical interface corresponding to that VLAN. Likewise, when the router sends a frame to the switch in response to a request it will add a VLAN tag to the frame such that the switch knows to which VLAN the response frame should be delivered. In effect, you've configured the router to "appear" as a physical device in multiple VLANs while only using a single physical connection between the switch and the router.
Routers on Sticks and Layer 3 Switches
Using virtual sub-interfaces you've been able to sell Internet connectivity to all your tenants without having to buy a router that has 25+ Ethernet interfaces. You're fairly happy with your IT accomplishments so you respond positively when two of your tenants come to you with a new request.
These tenants have opted to "partner" on a project and they want to allow access from client computers in one tenant's office (one given VLAN) to a server computer in the other tenant's office (another VLAN). Since they're both Customers of your Internet service it's a fairly simple change of an ACL in your core Internet router (on which there is a virtual sub-interface configured for each of these tenant's VLANs) to allow traffic to flow between their VLANs as well as to the Internet from their VLANs. You make the change and send the tenants on their way.
The next day you receive complaints from both tenants that access between the client computers in one office to the server in the second office is very slow. The server and client computers both have gigabit Ethernet connections to your switches but the files only transfer at around 45Mbps which, coincidentally, is roughly half of the speed with which your core router connects to its switch. Clearly the traffic flowing from the source VLAN to the router and back out from the router to the destination VLAN is being bottlenecked by the router's connection to the switch.
What you've done with your core router, allowing it to route traffic between VLANs, is commonly known as "router on a stick" (an arguably stupidly whimsical euphemism). This strategy can work well, but traffic can only flow between the VLANs up to the capacity of the router's connection to the switch. If, somehow, the router could be conjoined with the "guts" of the Ethernet switch itself it could route traffic even faster (since the Ethernet switch itself, per the manufacturer's spec sheet, is capable of switching over 2Gbps of traffic).
A "layer 3 switch" is an Ethernet switch that, logically speaking, contains a router buried inside itself. I find it tremendously helpful to think of a layer 3 switch as having a tiny and fast router hiding inside the switch. Further, I would advise you to think about the routing functionality as a distinctly separate function from the Ethernet switching function that the layer 3 switch provides. A layer 3 switch is, for all intents and purposes, two distinct devices wrapped up in a single chassis.
The embedded router in a layer 3 switch is connected to the switch's internal switching fabric at a speed that, typically, allows for routing of packets between VLANs at or near wire-speed. Analogously to the virtual sub-interfaces you configured on your "router on a stick" this embedded router inside the layer 3 switch can be configured with virtual interfaces that "appear" to be "access" connections into each VLAN. Rather than being called virtual sub-interfaces these logical connections from the VLANs into the embedded router inside a layer 3 switch are called Switch Virtual Interfaces (SVIs). In effect, the embedded router inside a layer 3 switch has some quantity of "virtual ports" that can be "plugged in" to any of the VLANs on the switch.
The embedded router performs the same way as a physical router except that it typically doesn't have all of the same dynamic routing protocol or access-control list (ACL) features as a physical router (unless you've bought a really nice layer 3 switch). The embedded router has the advantage, however, of being very fast and not having a bottleneck associated with a physical switch port that it's plugged into.
In the case of our example here with the "partnering" tenants you might opt to obtain a layer 3 switch, plug it into trunk ports such that traffic from both Customers VLANs reaches it, then configure SVIs with IP addresses and VLAN memberships such that it "appears" in both Customers VLANs. Once you've done that it's just a matter of tweaking the routing table on your core router and the embedded router in the layer 3 switch such that traffic flowing between the tenants' VLANs is routed by the embedded router inside the layer 3 switch versus the "router on a stick".
Using a layer 3 switch doesn't mean that there still won't be bottlenecks associated with the bandwidth of the trunk ports that interconnect your switches. This is an orthogonal concern to those that VLANs address, though. VLANs have nothing to do with bandwidth problems. Typically bandwidth problems are solved by either obtaining higher-speed inter-switch connections or using link-aggregation protocols to "bond" several lower-speed connections together into a virtual higher-speed connection. Unless all the devices creating frames to be routed by the embedded router inside the later 3 switch are, themselves, plugged into ports directly on the layer 3 switch you still need to worry about the bandwidth of the trunks between the switches. A layer 3 switch isn't a panacea, but it's typically faster than a "router on a stick".
Dynamic VLANs
Lastly, there is a function in some switches to provide dynamic VLAN membership. Rather than assigning a given port to be an access port for a given VLAN the port's configuration (access or trunk, and for which VLANs) can be altered dynamically when a device is connected. Dynamic VLANs are a more advanced topic but knowing that the functionality exists can be helpful.
The functionality varies between vendors but typically you can configure dynamic VLAN membership based on the MAC address of the connected device, 802.1X authentication status of the device, proprietary and standards-based protocols (CDP and LLDP, for example, to allow IP phones to "discover" the VLAN number for voice traffic), IP subnet assigned to the client device, or Ethernet protocol type.
Best Answer
I'm not sure if Windows Server 2008 R2 supports the network set up you've used there, neither in Virtual Machine Manager 2008 R2 nor in Hyper-V Manager on 2008 R2 do I see how to set VLAN trunking on a per-virtual machine level. In my set up, per Virtual Machine Manager 2008 R2's network configuration, I see this:
On the switch, I have VLAN 1 set as the primary (untagged) and everything else tagged and the switch ports for the server set to trunk. I then set up my Vyatta box so that it has a single NIC per VLAN I wanted to access, I was never able to set up a trunk on the VLAN and the management for the VM seem to indicate that it's not possible in 2008 R2, and I'm not sure if it's actually implemented for it.
I do know that the
Set-VMNetworkAdapterVlanPowerShell cmdlet was added in the WMF 3.0 management pack, which is advised not to be installed on 2008 R2 currently because it was released with serious issues. (WMF 4.0 is slated to be released soon and resolve those problems, I believe.) It might be possible that the cmdlet you're using to set the Vyatta VM's network adapter to trunked mode isn't fully implemented on 2008 R2. I've never seen a VM configured in trunked mode myself.Alas, I do not have a 2012 setup with Vyatta and Wireshark to test your setup.