Electrical – GPU vs CPU on chip memory

cpudramgpusram

There has been a fairly wide ranging discussion as to why it is difficult to combine memory / CPU logic on the same die (yield issue compounding, different processes, different clock-frequency, increasing testing requirements etc).

It seems that the latest NVDA GPU chips have >10GB of on-chip memory. These chips are also produced at leading edge nodes (i.e. 10nm below, TSMC process). So my question is: why does the memory / logic same die combination seem to work better for GPUs versus CPUs?

Best Answer

It seems that the latest NVDA GPU chips have >10GB of on-chip memory.

On-package != on-chip. The HBM are stacks of several chips right next to the main GPU. Consumer grade GPUs have the memory soldered to the PCB in normal BGA packages.

GPU memory works "better" because there are no connectors that could impact the signal path between chips, allowing higher frequencies.

PC main memory uses sockets, this allows you to upgrade the RAM by changing or adding modules. But these also degrade the electrical signals, and thus performance.

Also note that only a few MB on the GPU chips itself are SRAM (caches) - the overwhelming majority of memory is DRAM.