I am curious about NIC troughput vs PCIE slot speed.
I have found Mellanox MHQH19B-XTR which claims to be 40 Gbps PCI-E 2.0 x8 cards.
However, this slot limit seems to be theoretical ~ 31 Gbps.
So, what's going on?
How these speeds are acheived if I put it into router that's need to read all the packets? In my view they need to be transmitted via PCIE slot to be read by OS. Anyone can explain?
Best Answer
Such a card can be limited by its PCIe interface. PCIe 2.0 x8 is 40 gigatransfers per second, but with a 8b/10b line code that's 32 Gb/s. Not a problem for 2x 10Gb Ethernet, but perhaps a bottleneck for 40 Gb InfiniBand.
If you need to exceed 32 Gb/s, forget the older cards and get one with PCIe 3.0 or 4.0. PCIe 3.0 x8 is 63 Gb/s, which can easily support 2x 25 Gb Ethernet.