Looking at a list of the very latest CPUs, I see several of them with a cache size of 12MB or 8MB – pretty small, when compared to the ever-increasing size of hard drives and ram. It seems to be taken for granted that a cpu cache will always be small, but, why is that? Is it just economicaly unfeasible, or are there engineering reasons why it has to stay small?
I'm thinking of some cryptocurrencies (i.e. ethereum), that are designed to be memory-hard, so the speed of the algorithm is limited by the IO bandwidth of the memory, with the idea that this makes it impossible to design a custom chip specifically to solve that algorithm as was done for bitcoin. But, if someone was making a custom chip anyway, couldn't they just cram a gigabyte into the cache and do away with the IO bottleneck?
Best Answer
It's a trade-off between the higher hit rate of a large cache and the faster speed of the smaller cache RAM. Hit rate follows a law of diminishing returns as the cache gets larger.
Doubling the size of a large cache may only less than a couple of percent increase in hit rate but it'll certainly increase its access time, which slows CPU throughput.