Electronic – Does fly-by order of bytes matter in DDR3+ design

DDR3 has integrated (on-die) termination for the data lines, and Vtt termination for other lines. Therefore, not all lines require external termination resistors. You will find that several high-speed IO standards have integrated termination, as this avoid the parasitics, extra space, and extra routing of "regular" discrete termination.

I suggest you read this Altera DDR3 termination & layout guidelines document as well as this Wikipedia article: On-die termination and this presentation on DDR3/DDR3L/DDR4.

DDR3 termination is a very complicated subject, as different bits of the interface have different termination situations, as well as on-die termination which can also be dynamic (register settings can modify the value of the on-die termination). To better understand the nuances, you'll likely want to look at several DDR3 controller/DRAM datasheets and reference designs.

Finally, see TIDEP0012 for an example of a DDR3 reference design that does not use Vtt termination.

To sum up, DDR3 is somewhat of a special case when it comes to "conventional" high-speed digital designs, due to integrated features that make it easier to use. Ritchey's book is a good one, I would also suggest reading Howard Johnson's and Henry Ott's texts for more high-speed digital design considerations. Also, without looking at the schematics, it's difficult to fully understand the circuit at hand. Last I checked, the Raspi 3 was still closed hardware. Have you got a schematic? You'll have a far better time designing circuits by reading the datasheet and looking at reference designs than by trying to infer designs from closed hardware.

Your calculation of the peak throughput is correct, given the input.

The actual throughput largely depends on the design of the DMA and DRAM controllers. The 3.4 GB/s is a good approximation, but it relies on the assumption that the DMA controller does one 64-byte transfer at a time and each transfer hits already opened (active) DRAM bank.

In DRAM, reads and writes may be partially overlapping, so that the column address of the next operation is sent to the DRAM chip while it is still transferring data for the previous access. This is how actual throughput may get close to the theoretical peak. For this to happen, there should be enough buffer space to store the burst data, obviously. In memory to CPU transfers, usually the CPU cache can quickly consume or produce a relatively large chunk of bytes, when prefetching from memory. I am not sure about DMA controllers, though. Your DMA controller may have FIFO buffers, but I don't know if there are big enough to sustain a transfer longer than a single burst. Lacking enough buffers may be the reason you are not getting close to the theoretical throughput.

Another aspect is related to DRAM pages. In order to issue reads and writes, the corresponding page has to be open ("activated"). Typically, DRAM chips consist of several banks, each having its own page buffer. If source and destination of your transfers happen to reside in different banks, there may be no additional cost of opening a page. Otherwise, the controller may have to close and open pages on every access, which would slow down the transfers significantly (~30-50 ns each time). The controller may also choose to close the DRAM page after each access, implementing so called "closed page" policy, if configured to do so.

There will always be page open/close overhead when crossing pages, but it should be amortized in large sequential reads and writes.

Unfortunately, to get a more definitive answer, you have to dig through the DRAM and SoC/DMA controller datasheets, if they are available.

ADDED: I forgot to ask: why are you concerned about DMA memory to memory throughput? Since DMA is mostly intended for device access, it may not be designed for high speed and you may have better luck copying memory by CPU.