Recently I came across a new kind of NIC capable of 400Gbps (It kind of blew my mind).
Does it mean it's electric circuitry is capable of serializing/deserializing (SERDES) 400G bits per second of data onto a wire while maintaining a relativley clean signal (low SNR)?
So bandwidth is really the speed at which bits are serialized ,not traveled across, a link? Does it also indicate the time between 1 and 0 pulses? (But the whole serialized packet moves at the ~speed of light)
Or is it something else?
Thanks!
Best Answer
Yes, that's what 400GE is designed for. The physical coding sublayer (PCS) uses forward error correction (FEC) to achieve a block error rate of 10-13 or better. The acceptable SNR varies with the different PHYs.
400G Ethernet uses multiple 25, 50, or 100 Gbit/s lanes (with up to 53 GBd using PAM-4), so it requires multiple fiber strands or wavelengths (or differential pairs in a backplane). 400GBASE-LR4 currently provides up to 10 km reach (nominally), the upcoming 400GBASE-ER8 is going to support up to 40 km.
Propagation in fiber is generally limited by the fiber's velocity factor, ca. .67 (the reciprocal of the refractive index), so .67 x c0 ≈ 200,000 km/s. Accordingly, on the fiber the PAM-4 symbols at 53 GBd are 3.8 mm "long".
For Ethernet, the nominal bandwidth is present at the top of the physical layer. It includes "high-level" signaling like the preamble and inter-packet gap (IPG) and of course, L2 frame header and footer. However, it excludes line code overhead from PCS.
That way, the exact, usable bandwidth can be very easily calculated: for maximum-sized frames there's 1500 bytes L3 payload and 38 bytes overhead for L1 & L2 in total. Thus, 400GE provides a usable bandwidth for L3 of 400 Gbit/s / 8 bit/byte / 1538 bytes * 1500 bytes = 48,76 GB/s.
Incidentally, Broadcom just announced their new Tomahawk 4 switch chip, sporting 64x 400 Gbit/s ports (or 256x 100 Gbit/s) for a total backplane capacity of 25.6 Tbit/s. The chip with 31 billion transistors must be huge, even in 7 nm. Truly mindblowing...