Not all together happy with how this post turned out, but I've learned things that I didn't know before, so that's good.
For example, normally if two nodes (A and B) are connected to a switch which is on the downstream side of another device like your firewall, the switch will route packets from A -> B and vise versa without contacting anything upstream (from @Tedwin).
I decided to post my own answer with the good parts from the comments with @Panther Modern.
Use higher quality hardware. Since I'm using Netgear ProSafe unmanaged switches, I'm considering upgrading to something like Cisco. @Mike Pennington suggested the SG300 series, which I have researched before and found on Amazon for less than $200. @Panther Modern suggests Arista, Cisco, Juniper and Brocade switches.
Troubleshoot each piece independently and isolate the problem. I've done some of this, to the extent that I have isolated the general path, but in an environment where 100% uptime makes people happy and allows them to get work done, this can be tricky. But point taken...
Consider topology and where you have access to and how many users you have. These factors can influence your choices for gear and how you go about answering your questions.
Find good networking tools and use those to help troubleshoot your issue. pfSense helped a lot concerning traffic that passed through the firewall. I also learned about Robocopy for limiting traffic on the client. This tool works pretty well.
The three major differences between a managed switch and using Linux bridge interfaces are performance, port density, and features.
Most managed switches will embed some of their programmed functionality into specialized hardware. This is not true in all cases, but this specialized hardware will tend to outperform devices that are purely software based (at least for the functionality embedded in the hardware).
Second, if port density is of concern, there aren't many server systems where you can pack 12-48 ports into a 1U chassis, and of the ones I have seen they were designed to be a network device.
Finally, are the features. Managed switches will typically have features that either are not present on a Linux platform, not as easily configured, or may require additional CPU/memory resources that will further impact performance if you use them.
However, aside from the differences in the two platform choices, it sounds like you are setting up some sort of lab/test/dev environment. My primary concern would be that you should try to match your actual/production environment as closely as possible. Your Linux "switches" do not behave the same as your managed switches, so something you implement in the lab may act entirely differently when implemented on your managed switches.
Best Answer
Good question. The short answer: No, there's no inherent difference in the speed or latency available to hosts talking to one another on a managed vs. unmanaged switch. In reality though, you'll generally see better performance all around on a managed switch, because unmanaged switches are typically less capable in every sense when compared to a managed switch.
The long answer: Most switch manufacturers provide specifications that often include backplane speed, data rate, and throughput.
So the long answer really is, you need to compare specific switch models - look at the spec sheets and see for yourself!