Electronic – Lane Reversal on PLX PCIe Switches

layoutpcieswitches

I'm currently designing a carrier board on which is integrated a PLX PEX 8750 PCIe switch connected to several PCIe connectors.
The problem occurs when routing PCIe lanes, since I can't route them without having to “criss-cross” the lanes. Thereby, I consider using the lane reversal capability.

The PEX 8750 supports dynamic Lane reversal as mentioned on its datasheet, but it also indicates that "If the wiring of Lanes to a device is reversed (on both Transmitters and Receivers), only one of the two connected devices must support Lane reversal."

I also read a paper published by Intel which says: "Unless you are certain that lane reversal is supported and guaranteed by at least one of the devices in question, do not rely upon this feature."
Taking Advantage of the PCI Express Specification – EDN

So, which of the two sources should I believe?

Best Answer

According to section 4.2.4.10.1 of the PCIe 3.0 Base Specification:

Support for Lane reversal is optional.

• If implemented, Lane reversal must be done for both the Transmitter and Receiver of a given Port for a multi-Lane Link.

• An example of Lane reversal consists of Lane 0 of an Upstream Port attached to Lane N-1 of a Downstream Port where either the Downstream or Upstream device may reverse the Lane order to configure a xN Link.

So according to that second point, either the Downstream of Upstream device can reverse the lane order.

Later on, in section 4.2.6.3.2.2, it discusses example cases for lane reversal. In the main example, it is written:

A x8 Upstream Port is presented with Lane numbers that are backward from the preferred numbering. If the optional behavior of Lane reversal is supported by the Upstream Port, the Upstream Port transmits the same Lane numbers back to the Downstream Port.

Otherwise the opposite Lane numbers are transmitted back to the Downstream Port, and it will be up to the Downstream Port to optionally fix the Lane ordering or exit Configuration.

Interpreting this further, as long as at least one of the devices supports lane reversal a link can be established.

If lane reversal is needed, and the upstream device (closest to the RC) is capable of doing it, then it will always be the one to do the reversal regardless of whether or not the downstream device supports it.

The link process is something like:

  1. The downstream port tells the upstream port its preferred ordering.

  2. The upstream port will try to match that request

    1. If the upstream was able to match the lanes requirement, either by using its preferred layout, or by reversing the lanes. It will echo back to the downstream port the corresponding mapping.
    2. Otherwise if the ordering cannot achieved (i.e. upstream doesn't support lane reversal), then it is left to the downstream port to try to reverse the lanes. The upstream port will echo back a reversed mapping.

  3. The downstream port will check what lane mapping the upstream port is echoing.

    1. If the echo matches the preferred mapping, a link is established.
    2. Otherwise it is now up to the downstream port to reverse the lane mapping. If it supports lane reversal it will now tell the upstream port a reversed lane mapping and a link is established.
    3. If the downstream cannot reverse the lanes, a link cannot be established.
Related Topic