4. Cache - Bottleneck

The picture below shows the arrangement of a Von Neumann, stored program computer. This is the basis of modern sequential CPUs. Namely, both instructions and data are stored externally in memory and to get data and instructions into the CPU, it crosses the data bus.

We have discussed how external memory affects performance, but there is a more subtle problem as well. Namely all the data and instructions have to go across the data bus in order to be handled by the CPU.

But the transfer rate is far slower than the speed of the CPU.

This problem is called the 'von neumann bottleneck'.

If nothing was done about it, then the performance of the computer is limited by the data bus. Increasing the speed of the CPU just means it sits around even longer waiting for instructions and data. The next few pages explain how engineers have addressed this problem.