If I have a leaf-spine network with all the servers on the same subnet, say 10.10.0.0/16, do I need a router (or l3) since everything is on the same network and the network is flat?
Leaf-Spine Network – How to Implement Without Routing
layer3Networkroutingswitchvlan
Related Solutions
No, this would not work, each VLAN is a separate broadcast domain. Each VLAN has its own MAC address table which the switch uses to forward traffic at L2 between the ports that are in the same VLAN. Each VLAN also only contains the ports that are assigned to that VLAN.
Firstly the ARP from the PC in VLAN 100 would not be forwarded to the PC in VLAN 200 (separate broadcast domain), so the PC on 192.168.100.2 would not be able to resolve the MAC address of the PC at 192.168.100.3
Secondly, even if you created a static ARP on the first PC for the second PC, VLAN 100 does not contain the MAC address of the second PC.
If you tried to create a static MAC entry in the MAC table of VLAN 100 for the second PC you could not as the port the second PC is connected to is not in VLAN 100.
In order to communicate between VLANs, you need to configure routing. This can be done on a L3 switch or router. You would of course not be able to route between two networks with the same network address (on the same router) as Cisco routers do not allow you to configure the same network on two ports in the same VRF (Juniper routers may allow two interfaces on the same network in the same VRF, but this is not the norm).
The only way to do this would be to physically connect a port from VLAN 100 into a port from VLAN 200, to bridge the two VLANs together.
Of course, you would never do any of this in real life.
That isn't really a canonical spine-and-leaf network, at least not as typically conceived in current networks.
Take your diagram and delete the core nodes and all of their connections. One pair of leaves can be set aside as border leaves, meaning that they're responsible for external connectivity. In truth for most implementations there's no particular reason a given set of leaves can't both host end-node connectivity and provide external links, but this can vary.
The basic reason for this is that a given leaf has more bandwidth to every other leaf than does any given spine. Spines should be incredibly simple in their configuration and do little more than provide connectivity between leaves.
So.. this changes some of your questions, but I'll try to address them:
- The cores go away. Generally speaking there will be some kind of L3 connectivity between the spines and leaves. It could be static routes, but almost never will. It realistically needs to be a protocol that can support equal cost multipathing (ECMP) for greater than 2 nodes - which, in any kind of sane practice, means either an IGP or BGP. Both can be valid choices and, indeed, for several popular options both are used simultaneously (iBGP + an IGP).
- Leaf and spine communication needs to be L3. You may use an L2 overlay (i.e. VXLAN-EVPN) to provide L2 as a service, but the fundamental premise of spine-leaf is the use of something that is capable of ECMP - which, again, is not native L2. Even L2-only protocols like Cisco's FabricPath are actually encapsulations (in the FP case it's a mac-in-mac encapsulation that's using IS-IS to advertise nodes and associated addresses). In the EVPN case, VXLAN is used as a MAC-in-UDP encapsulation while BGP provides mapping between a given tenant's MAC and IP addresses and an associated endpoint.
There's also the case of L3 hosts that's a valid spine-leaf design. In this instance either the leaves are just advertising local networks (i.e. no L2 mobility between leaves, or leaf-pairs) or the hosts themselves advertise the local addresses of VM's, containers or loopbacks. In practice this means a bunch of /32's (or /128's if you're v6) being pushed via local subnets on the various leaves.
Best Answer
Technically, no - if all nodes reside in that flat network.
However, such a large subnet is not good practice due to limited scaling and the potential propagation of any L2 problems.
The core-distribution links should always be routed (L3) instead of switched. In current practice, the distribution-access links are increasingly becoming L3 as well which provides even better scalability. Most often, L3 switches are used between core and distribution, and between distribution and access.
With a smaller network (and subnet) you would use a collapsed core topology where the access switches connect to the core directly. Again, good practice is to route those links.
Using routed instead of bridged links can improve total scalability (when it's not practical to propagate each MAC address throughout the whole network), total throughput (in contrast to STP blocking redundant links you can use equal-cost multi-path routing) and resilience (a complex network can fail over quicker on a link-state routing protocol than by R/MSTP).
Shortest Path Bridging heavily borrows from ECMP and scales significantly better in an L2 scenario. However, the industry hasn't quite caught on yet with standard switches, so L3 with ECMP is currently a better and more future-proof way.
All in all, L3 vs L2 depends on the level of scalability you have in mind. A network with a few hundred nodes that is not growing (are you sure?) usually works well in a flat L2 design.