Why does iSim give a different result than hardware

simulationspartanverilog

I am working on a MIPS CPU for an FPGA – this is mostly a personal project to understand FPGA's.

I have a 5 stage pipeline CPU implementation working correctly when run on iSim, however when I run it directly on the hardware (Spartan 6) I get a different result.

The test program I have is simple:

load 0x128 into reg[1]   
nop*
load 0x160 into reg[2]
nop*
add reg[1] to reg[2]
assert "finished" flag

when I run this in iSim, I get the correct result in reg[2] (0x288) however, when I run the program on the FPGA I get the result 0x3b0 – it appears that the final instruction is running multiple tiems

The nop instructions are present to force a cache miss during execution, and this is what causes the problem – without these the result is correct.

What I think is happening is that I assert a "stall" flag at the positive edge of the clock cycle – and something else that runs at the positive edge relies on the presence/absence of this flag. iSim by chance runs correctly (due to the arbitrary order in which it executes the simulation of always blocks) however the FPGA fails as the stall change is not available until the negative edge.

I have already scoured my verilog implementation to find where this is – and am still looking. My question is: how can I force iSim to expose this error?

Best Answer

The issue was caused by RAMB16WER reading data on the positive edge. My fetch code was changing these signals on the positive edge too. Moving this code to the negative always block fixed it.

Its odd that it worked up to a certain point on the device, and that it worked fully on iSim, but I guess this is due to the way iSim executes the always blocks.

Related Solutions

Reading a serial data stream with Verilog

You have two requirements here:

You need to make sure you sample that data at the correct time according to the bus needs.
You need to synchronize this data into your system clock domain.

There are a couple of ways to meet these needs.

First, if the bus clock is slow enough below your system clock, then you can synchronize the bus clock to your clock domain with a double flop. Then use a simple edge detector to determine when the rising edge is. This is used to safely sample the data line of the bus. Note that this achieves #2 automatically. Also, as noted here, this is the preferred way to do this in an FPGA.

This method has a drawback in that the data is sampled somewhere between two and three system clocks later than the actual 'on the pin' bus clock edge. If this is too long (due to your system clock not being fast enough in comparison), you have to go an alternative way.

In this method, you sample before synchronizing to the system clock domain. The reason is to make sure you are sampling at the right time according to the bus.

always @(posedge bclk) begin  // positive edge of bus clock
    sampled_data <= data;
end

At this point, you have sampled_data which is a signal in the bclk domain. You need to synchronize it to your system clock domain. To do this, you have to use handshaking or a FIFO.

One way that works is to do the shift register in the bus clock domain to get to parallel data. Then pass it through a dual clock FIFO to the other domain. FPGAs have primitives just for this use.

// Latch the data in the bclk domain
always @(posedge bclk) begin
    fifo_d <= {fifo_d[30:0], data};

    data_count <= data_count + 1;
    if (data_count == 31)
        fifo_wr <= 1'b1;  // This will latch fifo_d to the dual clock fifo input
    else
        fifo_wr <= 1'b0;
end

// Read the data out of the FIFO in the system clock domain.
always @(posedge clk) begin
    if (fifo_ready)
        synchronized_data <= fifo_q;   // Now the data is in your domain.
end

Other notes:

As with any I/O at the edge of the FPGA as well as between clock domains, you will need to correctly define the timing constraints. Do describe the details is a bit too general for this forum, but as recommended in the comments by Greg, this paper is a good source for understanding the needs for the clock domain crossing. The FPGA vendors tend to have decent write ups for input delay and output delay definitions as well.

Verilog, register values

So after a lot of digging I found the answer to the problem. I am not sure if it helps anyone but I am going to post the answer anyways.

Here are the signals of the code above failing.

Processor failing

The problem is actually very simple and I only spotted it when someone told me this: "At the positive edge of the clock something should change, otherwise your design is stuck!". If you look at the signals above, at 15ns something is wrong because none of the registers change when the processor should have executed the first instruction! The reason is because the signel instr[7:0] didnt get assigned the value of the next instruction (which was correctly fetched). The problem is because of this section of code:

fetch.v:

always @ (posedge clk) begin
    if (exec_stat == 1'b1) begin
            // If we are fetching, store the instruction in the 'data' buffer and set exec_stat low 
            exec_stat <= 1'b0;
            empty_instr[3:0] <= 4'b0000;
            instr0 <= data[7:0];
            instr1 <= data[15:8];
            instr2 <= data[23:16];
            instr3 <= data[31:24];
    end else if (iptr[1:0] == 2'b00) begin
        // the last two bits of the iptr address define which 8-bits from the buffer we need to use as instruction
        if (empty_instr[0]) begin
            // if there is no instruction at this address then
            // go to fetch state
            exec_stat <= 1'b1;
        end else begin
            instr <= instr0;
            empty_instr[0] <= 1'b1;
        end 
    end else if (iptr[1:0] == 2'b01) begin
        if (empty_instr[1]) begin
            exec_stat <= 1'b1;
        end else begin
            instr <= instr1;
            empty_instr[1] <= 1'b1;
        end 
    end else if (iptr[1:0] == 2'b10) begin
        if (empty_instr[2]) begin
            exec_stat <= 1'b1;
        end else begin
            instr <= instr2;
            empty_instr[2] <= 1'b1;
        end 
    end else if (iptr[1:0] == 2'b11) begin
        if (empty_instr[3]) begin
            exec_stat <= 1'b1;
        end else begin
            instr <= instr3;
            empty_instr[3] <= 1'b1;
        end 
    end 
end

This is wrong because it means that 'instr' is driven when the clock goes high, which means that there are some cycles that the processor does nothing and the PC is not updated in time...

The solution is to NOT make 'instr' a registered output rather constantly assign it using a multiplexer. Something like:

mux8_4 mux0_4(iptr[1:0], instr0, instr1, instr2, instr3, instr);

After all, if it contains the wrong instruction it doesnt matter because that means that we would go into a fetch state and the processor would ignore the values. So there is no reason to put this assignment in the always block. With this change the signals look like this:

correct

Clearly, everything now changes at the appropriate time!!!! :) :D

Best Answer

Related Solutions

Reading a serial data stream with Verilog

Verilog, register values

Related Topic