I'm looking to build a simple board that has a 16x PCIe connector, that seats a compatible PCIe video card.
I can find the connectors and create a footprint, on the board. I can find and understand to some degree the basic pin-out of the PCIe 1x connector — but I'm not clear on the full end to end implementation.
I've read the full specifications costs lots of money, and honestly is overkill for what I want.
I'm not super concerned about the actual protocol (like at the physical layer), so not necessarily looking for the official PCIe spec — just the nuts and bolts implementation. Which pins go where on the cards, where they are laid out.
I am not certain on signal routing both between the mating connector and the PCIe card, and then where the card signals.
I know Altium provides templates of the actual pads and so on, but I'm more interested in where the signals go, and what each one does, and which signals need to be treated as high-speed signals and so on.
It seems like this kind of information isn't so open-source? It's not like a video manufacturer is just posting up a datasheet for every person who buys a PCIe card to understand how it is routed from card to the eventual CPU on a motherboard.
For people who are building video cards or whatever for PCIe, where do they learn how to build these devices?
Best Answer
PCIe has "End Points (EP)" and a "Root Complex (RC)". The GFX card in this case is known as an EP which is basically like a leaf of the tree. At the trunk of the tree would be the CPU on your motherboard which has an RC built in to it which manages all devices.
Wiring between an EP and an RC is at it's simplest just point to point. Within this there are two types of signals - lanes (TX and RX), and sideband signals (everything else, e.g. clock, reset, power, etc.). Of these signals the following should be noted:
For the sideband signals, these should pretty much all just be connected between corresponding pins at either end. In your case for a riser card you'd simply pass them straight from the pin on one connector to the same pin on the other.
Power traces should be routed with sufficient with to supply either 25W or 75W. There is generally a 12V supply, a 3.3V supply, and an auxiliary (standby) 3.3V supply when the PC is asleep or powered off.
The lane signals are the most critical. These must be routed with great attention to detail ensuring they are differentially routed and impedance matched to 100 Ω differential. This is typically achieved by using a ground plane on a 4-layer board as a reference plane (HF routing is beyond the scope of this answer, there are plenty of EE.SE resources here). You must length match the differential pairs P/N signals for each lane, however you don't need to worry about matching the lengths of different lane for PCIe as each lane is handled separately from the rest anyway.
Lanes are unidirectional - i.e. the data flows one way. It is imperative to cross-over TX and RX lanes - the TX lane of one device goes to the RX lane of the other device. However, if you are building a riser card, the cross-over of the lanes is done on the motherboard, which means that your riser should routed the signals from one connector to the same pin number on the other end (TX on one end goes to TX on the other end).
If you are changing the number of lanes, you need only wire up the lanes you require ensuring that the lowest number contiguous chunk of lanes (e.g. Lane 0 for 1x, Lanes 0 & 1 for 2x, etc.) are kept. PCIe expects that lane 0 on one device connects to lane 0 on the other device always (*) and that no lanes are skipped. So if you are connecting your x16 card to a x1 slot you must keep lane 0 and need only wire that one up. Any unused lanes can be simply left disconnected (you don't need to nor should you terminate them).
Beyond that it is just a case of following good PCB practice and taking special care with impedance matching and high frequency trace routing.
The part which gets very tricky is if you hope to get Gen 3 PCIe (and Gen 2 to a certain extent). The speeds at which G3 runs (4GHz) are far beyond the capabilities of standard FR4 PCB substrates and routing them takes you into the weird and wonderful world of RF design - think Wi-Fi radio signals being channelled through PCB traces and you realise how much of a pain it is. Gen 1 is only (o_O) 1.25GHz which is a little bit more forgiving as long as you are neat with your routing and try to avoid vias where possible.
(*) Strictly speaking there are other clever tricks like lane reversal but support is not mandated.
With regards to the final question of where do people learn this stuff - generally while working for companies or research institutions. There aren't a lot of resources out there because designing PCIe devices is generally far beyond what most hobbyists would attempt, and the implementation information is probably kept as corporate secrets by those who make commercial devices.