Cache Memory Principles | Computer Architecture & Organisation (CAO) - Computer Science Engineering (CSE) PDF Download

Principles

• Intended to give memory speed approaching that of fastest memories available but with large size, at close to price of slower memories
• Cache is checked first for all memory references.
• If not found, the entire block in which that reference resides in main memory is stored in a cache slot, called a line
• Each line includes a tag (usually a portion of the main memory address) which identifies which particular block is being stored
• Locality of reference implies that future references will likely come from this block of memory, so that cache line will probably be utilized repeatedly.
• The proportion of memory references, which are found already stored in cache, is called the hit ratio.
• Cache memory is intended to give memory speed approaching that of the fastest memories available, and at the same time provide a large memory size at the price of less expensive types of semiconductor memories. There is a relatively large and slow main memory together with a smaller, faster cache memory contains a copy of portions of main memory.
• When the processor attempts to read a word of memory, a check is made to determine if the word is in the cache. If so, the word is delivered to the processor. If not, a block of main memory, consisting of fixed number of words is read into the cache and then the word is delivered to the processor.

• The locality of reference property states that over a short interval of time, address generated by a typical program refers to a few localized area of memory repeatedly. So if programs and data which are accessed frequently are placed in a fast memory, the average access time can be reduced. This type of small, fast memory is called cache memory which is placed in between the CPU and the main memory.

• When the CPU needs to access memory, cache is examined. If the word is found in cache, it is read from the cache and if the word is not found in cache, main memory is accessed to read word. A block of word containing the one just accessed is then transferred from main memory to cache memory.

• Cache connects to the processor via data control and address line. The data and address lines also attached to data and address buffer which attached to a system bus from which main memory is reached.
• When a cache hit occurs, the data and address buffers are disabled and the communication is only between processor and cache with no system bus traffic. When a cache miss occurs, the desired word is first read into the cache and then transferred from cache to processor. For later case, the cache is physically interposed between the processor and main memory for all data, address and control lines.

• CPU generates the receive address (RA) of a word to be moved (read).
• Check a block containing RA is in cache.
• If present, get from cache (fast) and return.
• If not present, access and read required block from main memory to cache.
• Allocate cache line for this new found block.
• Load bock for cache and deliver word to CPU
• Cache includes tags to identify which block of main memory is in each cache slot

Locality of Reference

• The reference to memory at any given interval of time tends to be confined within a few localized area of memory. This property is called locality of reference. This is possible because the program loops and subroutine calls are encountered frequently. When program loop is executed, the CPU will execute same portion of program repeatedly. Similarly, when a subroutine is called, the CPU fetched starting address of subroutine and executes the subroutine program. Thus loops and subroutine localize reference to memory.
• This principle states that memory references tend to cluster over a long period of time, the clusters in use changes but over a short period of time, the processor is primarily working with fixed clusters of memory references.

Spatial Locality 

• It refers to the tendency of execution to involve a number of memory locations that are clustered.
• It reflects tendency of a program to access data locations sequentially, such as when processing a table of data.

Temporal Locality

• It refers to the tendency for a processor to access memory locations that have been used frequently. For e.g. Iteration loops executes same set of instructions repeatedly.