Short Notes: Memory Hierarchy | Short Notes for Computer Science Engineering - Computer Science Engineering (CSE) PDF Download

 Page 1


Increased in capacity
and access time
Memory Hierarchy
• The memory unit is used for storing programs and data. It fulfills the need of 
storage of the information.
• The additional storage with main memory capacity enhance the performance 
of the general purpose computers and make them efficient.
• Only those programs and data, which is currently needed by the processor, 
reside in main memory. Information can be transferred from auxiliary memory 
to main memory when needed.
Cache Memory
• A small, fast storage memory used to improve average access time Or We can 
say that cache is a very high-speed memory that is used to increase the speed 
of processing by making current programs and data available to the CPU at a 
rapid rate.
• The cache is used for storing segments of programs currently being executed 
in the CPU and temporary data frequently needed in the present calculations.
https://gradeup.co/memory-hierarchy-i-30fB2e2d-c034-11e5-a349-4593a714a37b 1/6
Page 2


Increased in capacity
and access time
Memory Hierarchy
• The memory unit is used for storing programs and data. It fulfills the need of 
storage of the information.
• The additional storage with main memory capacity enhance the performance 
of the general purpose computers and make them efficient.
• Only those programs and data, which is currently needed by the processor, 
reside in main memory. Information can be transferred from auxiliary memory 
to main memory when needed.
Cache Memory
• A small, fast storage memory used to improve average access time Or We can 
say that cache is a very high-speed memory that is used to increase the speed 
of processing by making current programs and data available to the CPU at a 
rapid rate.
• The cache is used for storing segments of programs currently being executed 
in the CPU and temporary data frequently needed in the present calculations.
https://gradeup.co/memory-hierarchy-i-30fB2e2d-c034-11e5-a349-4593a714a37b 1/6
Memory connection in computer system
Cache Performance
When the processor needs to read or write to a location in main memory, it first 
checks whether a copy of that data is in the cache. If so, the processor immediately 
reads from or writes to the cache.
Cache hit If the processor immediately reads or writes the data in the cache line. 
Cache miss If the processor does not found the required word in cache, then cache 
miss has occurred.
Hit ratio Percentage of memory accesses satisfied by the cache.
Miss ratio = 1 - Hit ratio
• Temporal Locality: The word referenced now is likely to be referenced again 
soon. Hence it is wise to keep the currently accessed word handy for a while.
• Spacial Locality: Words near the currently referenced word are likely to be 
referenced soon. Hence it is wise to prefetch words near the currently 
referenced word and keep them handy for a while.
• Write through writes the data to memory as well as to the cache.
• Write back: Don't write to memory now, do it later when this cache block is 
evicted.
Main Memory
• The main memory refers to the physical memory and it is one central storage 
unit in a computer system.
• The main memory is relatively large and fast memory used to store programs 
and data during the computer operation.
• The main memory in a general purpose computer is made up of RAM 
integrated circuit.
Latency: The latency is the time taken to transfer a block of data either from main 
memory or caches.
• As the CPU executes instructions, both the instructions themselves and the 
data they operate on must be brought into the registers, until the 
instruction/data is available, the CPU cannot proceed to execute it and must 
wait. The latency is thus the time the CPU waits to obtain the data.
• The latency of the main memory directly influences the efficiency of the CPU.
Auxiliary-Memory: The common auxiliary memory devices used in computer 
systems are magnetic disks and tapes.
Magnetic Disks
• A magnetic disk, is a circular plate constructed of metal or plastic coated with 
magnetised material.
Page 3


Increased in capacity
and access time
Memory Hierarchy
• The memory unit is used for storing programs and data. It fulfills the need of 
storage of the information.
• The additional storage with main memory capacity enhance the performance 
of the general purpose computers and make them efficient.
• Only those programs and data, which is currently needed by the processor, 
reside in main memory. Information can be transferred from auxiliary memory 
to main memory when needed.
Cache Memory
• A small, fast storage memory used to improve average access time Or We can 
say that cache is a very high-speed memory that is used to increase the speed 
of processing by making current programs and data available to the CPU at a 
rapid rate.
• The cache is used for storing segments of programs currently being executed 
in the CPU and temporary data frequently needed in the present calculations.
https://gradeup.co/memory-hierarchy-i-30fB2e2d-c034-11e5-a349-4593a714a37b 1/6
Memory connection in computer system
Cache Performance
When the processor needs to read or write to a location in main memory, it first 
checks whether a copy of that data is in the cache. If so, the processor immediately 
reads from or writes to the cache.
Cache hit If the processor immediately reads or writes the data in the cache line. 
Cache miss If the processor does not found the required word in cache, then cache 
miss has occurred.
Hit ratio Percentage of memory accesses satisfied by the cache.
Miss ratio = 1 - Hit ratio
• Temporal Locality: The word referenced now is likely to be referenced again 
soon. Hence it is wise to keep the currently accessed word handy for a while.
• Spacial Locality: Words near the currently referenced word are likely to be 
referenced soon. Hence it is wise to prefetch words near the currently 
referenced word and keep them handy for a while.
• Write through writes the data to memory as well as to the cache.
• Write back: Don't write to memory now, do it later when this cache block is 
evicted.
Main Memory
• The main memory refers to the physical memory and it is one central storage 
unit in a computer system.
• The main memory is relatively large and fast memory used to store programs 
and data during the computer operation.
• The main memory in a general purpose computer is made up of RAM 
integrated circuit.
Latency: The latency is the time taken to transfer a block of data either from main 
memory or caches.
• As the CPU executes instructions, both the instructions themselves and the 
data they operate on must be brought into the registers, until the 
instruction/data is available, the CPU cannot proceed to execute it and must 
wait. The latency is thus the time the CPU waits to obtain the data.
• The latency of the main memory directly influences the efficiency of the CPU.
Auxiliary-Memory: The common auxiliary memory devices used in computer 
systems are magnetic disks and tapes.
Magnetic Disks
• A magnetic disk, is a circular plate constructed of metal or plastic coated with 
magnetised material.
• Often, both sides of the disk are used and several disks may be stacked on 
one spindle with read/write heads available on each surface.
• All disks rotate together at high speed. Bits are stored in the magnetised 
surface in spots along concentric circles called tracks. The tracks are 
commonly divided into sections called sectors.
Magnetic Tapes
• A magnetic tape is a medium of magnetic recording, made of a thin 
magnetisable coating on a long, narrow strip of plastic film.
• Bits are recorded as magnetic spots on the tape along several tracks. 
Magnetic tapes can be stopped, started to move forward or in reverse. 
Read/write heads are mounted one in each track, so that data can be recorded 
and read as a sequence of characters.
Memory Hierarchial Design
Level Name Registers
Cache
Memory
Main Memory
Secondary
memory
Typical Size <1 K B <16MB <16GB >100GB
Implementation
Customized
Multiports
SRAM (Flip- 
Flops)
DRAM
(Capacitor)
Magnetic
devices
Bandwidth
(MB/s)
(20,000-
1,00,000)
(5,000-
10,000)
(1,000-5,000) (20-150)
Access Time (ns) (0.25-0.5) (0.5-2.5) (80-250) 5000000
Managed By Compiler Hardware OS OS
Backed By
Cache
Memory
Main
Memory
Secondary
Memory
Compact Disk
Memory Organisation
Based on the style of accessing the memory, it can be classified into two types:
• Simultaneous Memory Access Organisation.
• Hierarchial Memory Access Organisation.
Simultaneous Access Memory Organisation
• In this memory Organisation, CPU is directly connected to all the levels of 
memory but accessing is allowed in a sequence i.e., Whenever there is a miss 
in level 1 memory, data can be accessed directly from level 2 without copying 
it into level 1.
• Levels are designed in such a way that lower the level, higher the 
performance, lower the access time and higher the level, lower the 
performance, and higher the access time.
• Hit ratio of last level must be 1 since it represent the whole system.
Page 4


Increased in capacity
and access time
Memory Hierarchy
• The memory unit is used for storing programs and data. It fulfills the need of 
storage of the information.
• The additional storage with main memory capacity enhance the performance 
of the general purpose computers and make them efficient.
• Only those programs and data, which is currently needed by the processor, 
reside in main memory. Information can be transferred from auxiliary memory 
to main memory when needed.
Cache Memory
• A small, fast storage memory used to improve average access time Or We can 
say that cache is a very high-speed memory that is used to increase the speed 
of processing by making current programs and data available to the CPU at a 
rapid rate.
• The cache is used for storing segments of programs currently being executed 
in the CPU and temporary data frequently needed in the present calculations.
https://gradeup.co/memory-hierarchy-i-30fB2e2d-c034-11e5-a349-4593a714a37b 1/6
Memory connection in computer system
Cache Performance
When the processor needs to read or write to a location in main memory, it first 
checks whether a copy of that data is in the cache. If so, the processor immediately 
reads from or writes to the cache.
Cache hit If the processor immediately reads or writes the data in the cache line. 
Cache miss If the processor does not found the required word in cache, then cache 
miss has occurred.
Hit ratio Percentage of memory accesses satisfied by the cache.
Miss ratio = 1 - Hit ratio
• Temporal Locality: The word referenced now is likely to be referenced again 
soon. Hence it is wise to keep the currently accessed word handy for a while.
• Spacial Locality: Words near the currently referenced word are likely to be 
referenced soon. Hence it is wise to prefetch words near the currently 
referenced word and keep them handy for a while.
• Write through writes the data to memory as well as to the cache.
• Write back: Don't write to memory now, do it later when this cache block is 
evicted.
Main Memory
• The main memory refers to the physical memory and it is one central storage 
unit in a computer system.
• The main memory is relatively large and fast memory used to store programs 
and data during the computer operation.
• The main memory in a general purpose computer is made up of RAM 
integrated circuit.
Latency: The latency is the time taken to transfer a block of data either from main 
memory or caches.
• As the CPU executes instructions, both the instructions themselves and the 
data they operate on must be brought into the registers, until the 
instruction/data is available, the CPU cannot proceed to execute it and must 
wait. The latency is thus the time the CPU waits to obtain the data.
• The latency of the main memory directly influences the efficiency of the CPU.
Auxiliary-Memory: The common auxiliary memory devices used in computer 
systems are magnetic disks and tapes.
Magnetic Disks
• A magnetic disk, is a circular plate constructed of metal or plastic coated with 
magnetised material.
• Often, both sides of the disk are used and several disks may be stacked on 
one spindle with read/write heads available on each surface.
• All disks rotate together at high speed. Bits are stored in the magnetised 
surface in spots along concentric circles called tracks. The tracks are 
commonly divided into sections called sectors.
Magnetic Tapes
• A magnetic tape is a medium of magnetic recording, made of a thin 
magnetisable coating on a long, narrow strip of plastic film.
• Bits are recorded as magnetic spots on the tape along several tracks. 
Magnetic tapes can be stopped, started to move forward or in reverse. 
Read/write heads are mounted one in each track, so that data can be recorded 
and read as a sequence of characters.
Memory Hierarchial Design
Level Name Registers
Cache
Memory
Main Memory
Secondary
memory
Typical Size <1 K B <16MB <16GB >100GB
Implementation
Customized
Multiports
SRAM (Flip- 
Flops)
DRAM
(Capacitor)
Magnetic
devices
Bandwidth
(MB/s)
(20,000-
1,00,000)
(5,000-
10,000)
(1,000-5,000) (20-150)
Access Time (ns) (0.25-0.5) (0.5-2.5) (80-250) 5000000
Managed By Compiler Hardware OS OS
Backed By
Cache
Memory
Main
Memory
Secondary
Memory
Compact Disk
Memory Organisation
Based on the style of accessing the memory, it can be classified into two types:
• Simultaneous Memory Access Organisation.
• Hierarchial Memory Access Organisation.
Simultaneous Access Memory Organisation
• In this memory Organisation, CPU is directly connected to all the levels of 
memory but accessing is allowed in a sequence i.e., Whenever there is a miss 
in level 1 memory, data can be accessed directly from level 2 without copying 
it into level 1.
• Levels are designed in such a way that lower the level, higher the 
performance, lower the access time and higher the level, lower the 
performance, and higher the access time.
• Hit ratio of last level must be 1 since it represent the whole system.
Average Memory Access time= H-|Ti+ (1-Hi )H2T2+ (1-Hi )(1-H2)H3T3+....
Where, Hi, H2, H3..are the hit ratio for level 1, level 2, Level3 and so on.
Hierarchial Memory Access Organisation
• In this memory organisation, CPU always access the data from the first level 
of memory. Whenever there is a miss in level 1, data is transferred from higher 
level of memory to the lower level of memory. After transfer of data, CPU 
access the data from lower level.
• It is the better approach if spatial locality is needed in the program.
Average Access time= H1 T 1 + (1-H1 )H2(T2+ T-,)+ (1-H1 )(1-H2)H3(T3+ T2+Ti ) +...