I'm wondering about the fact, that PCI architecture is hierarchical. Therefore, even if I have two PCIx4 slots, it's possible that I will not be able to fully utilize it, because the slots will connect in one node, which can have bandwidth insufficient to handle 2x PCIex4.
The background for my question is: I'm trying to utilize eight PCIe 1GBit interfaces. I have two cards packed with 2 ports, and one card packed with 4 ports. I'm able to get the maximum on 4 NIC interfaces. After activating 5th port, the performance slightly drops on each five interfaces. The same after activating 6th, 7th and 8ht interface.
Main question is: How to obtain PCIe structure on a machine, "draw" it, see it's nodes and connections, and deduce weakest node in that tree?
Best Answer
Each PCIe (v1) lane should comfortably handle 2 x 1Gbe links.
At this point DDR2 can comfortably handle the data rate of 10Gbe.
In general the PCIe layout of a machine is dictated by the chipset, and if you can find a diagram from Intel or whoever you should be able to work out where any bottlenecks are.
For example: