For a certain cache, these are the measurements:
Test of different sizes of cache memory, from 16 to 1024 words, Block size = 2 words and assosiativity = 1. Access time: 20 cycles.
Size 16 words 32 words 64 words 128 words 256 words 512 words 1024 words
Hit rate 1% 1% 1% 1% 1% 17% 50%
Cycle count 44841 44521 43881 42601 40041 34841 19481
Why does hit rate look like it does? Why is the rate so bad until the size reaches a certain size?
Best Answer
The "certain size" is known as the "working set" of a piece of code, and the size of the working set varies considerably among different pieces of code.
The question is whether or not a particular item in the cache gets replaced by something else before it can be reused. You only get a decent hit rate if the cache is large enough to hold most of the working set without replacement occurring.