5. Cache - Location

To partially overcome the 'von neumann bottleneck mentioned earlier, a special kind of memory called a 'Cache' is used. This sits on the CPU side of the data bus.

Imagine the data bus as a narrow bridge that can only carry so many instructions at a time. But what if we designed a 'holding area' on the CPU side of the bridge? Then we could store the most often-used instructions in the holding area instead of having to cross the bridge every time.

This holding area is called a 'cache' (pronounced 'cash'). If the software programmer is skilled enough, they will make it easier for the CPU to store the most-often used part of the code in the 'cache'. This activity is called 'code optimisation'.

To illustrate what is meant by 'often-used', consider the small piece of pseudo code below:

The instruction 'Add 1 to a variable' is repeated a 100 times in this loop, resulting in 100 transfers across the data bus. But if that instruction was stored in the cache, there is no need to use the slow data bus. Performance is improved significantly.

So the arrangement now looks like this:

This idea can be extended even further which is described on the next page.