I have an FPGA board with two Virtex-2Pro FPGAs on it. I try to send data from FPGA1 to FPGA2 and then send it back from FPGA2 to FPGA1. I use a register (always block in verilog) to drive each of out pins and a GPIO pin for probing.
I observe a 7-8ns delay between the two FPGAs by probing the GPIO pin using oscilloscope and my clock speed is 24MHz.
How do I reduce this delay? Is this long delay mainly due to the wire between FPGAs or is it because of the capacitance added by the GPIO pin and the probe?
Best Answer
The long delay is probably due to the internal structure of an FPGA. It's a fairly complex device.
Inside the FPGA, you have some logic that "does something". Let's say this something changes an output from 0 to 1. First, that output has to propagate through the connection matrix of the FPGA. Depending on where the map/place/route placed your logic, this could very well be across the entire FPGA! Especially given the 24 MHz clock and you probably didn't provide any other timing constraints, the toolchain will do what would appear to a normal human to be silly things during map/place. It could even use some CLBs to help propagate the signal, depending on what choices the toolchain made, but that usually happens only in large designs when the router starts to run out of resources.
Once it gets to the IOB, it still has to filter through all sorts of other circuitry before it gets out onto the pad. Once out of the pad, for every 6" of trace it will add about 1ns to the propagation delay. Then you have another trip inside the other FPGA, again traveling through the IOB and connection matrix. Any registers will also potentially add delay because the signal will have to wait for a clock edge.
EDIT:
The o-scope lead capacitance is not a factor in propagation delay (it doesn't change the permittivity of the traces etc). What it will affect is signal rise time. You can measure this with your scope. If you think the rise time is too slow, you can also set attributes on the IOB in the constraints file which will increase the slew rate and drive strength, at the expense of signal integrity. Note that even at 24 MHz, signal integrity can be an issue because it is the rise time of a signal and not the frequency itself that determines its spectral content. Drive strength isn't as hard on signal integrity, but if you start switching a lot of high drive strength IOBs at the same time (e.g. an address bus that goes from all 1s to all 0s), it can lead to ground bounce.
I see in the comments that you're having trouble with communicating between FPGAs. You should consider using some sort of handshaking so that FPGA2 knows when FPGA1 is sending data. If the FPGAs have different clocks, then you're also crossing clock domains, and that will cause problems because the signals are asynchronous to each other, meaning some of your flip flops will go meta-stable.
Consider two signals, Req (Request) and Ack (Acknowledge). If FPGA1 is the "master", have it put the data out on the bus, and raise the Req signal. Then, FPGA2 will see Req is high, and it will sample the data, and then raise the Ack signal. FPGA1 will see the Ack signal, and then lower Req, and importantly FPGA1 should also wait for Ack to get lowered before it decides to send more data.