Processors use both data and instruction caches in order to reduce the number of slow accesses to main memory. However, while it is clear to me that the data cache's purpose is to store frequently used data items (such as elements in an array or inside a loop), I cannot see what exactly the instruction cache stores that helps alleviate memory access times.
In the image above, we have an example of an "addi" instruction which adds a constant value to the value stored in general purpose register "r2" and writes the result to general purpose register "r1".
After this instruction is executed, what exactly is saved to the cache?
- It can't just be the opcode – most CPU instruction sets contain just a few hundred unique opcodes or less, so if the instruction cache was pre-loaded with all possible opcodes, it would always have a 100% hit rate. However, that would defeat the purpose of having a cache, plus I've read that instruction cache misses are very much possible.
- It can't be the values from main memory which are loaded into the general purpose registers, since that's exactly what the data cache is for.
Thank you in advance.