From what I understand, in a pipelined CPU, every stage takes 1 cycle. But instructions are fetched from memory which takes up to ~150 cycles. The CPU fetches most instructions from the L1-cache, but I've read that it takes around 4 cycles. From that logic, a new instruction should only start every 4th cycle, which obviously makes no sense?
This diagram shows a new instruction starts every cycle.
So, how does the CPU fetch a new instruction every cycle, if it takes more than one cycle to even fetch an instruction?
Best Answer
It fetches the next instruction, so they are processed in parallel. Note that next instruction could also fetch the result of the instruction that isn't finished yet. Of course, this is unwanted feature, but it can be triggered by writing the assembler code by hand. The compiler has to keep track on how these instructions are processed and to use the result only after write output (WO).