Design and Development of Effective Multi-Level Cache Memory Model

: An algorithm to determine the effectiveness and efficiency of a multi-level cache was developed in this paper. The developed model was used to test the efficiency rate, the relationships and the performance output level of a computer concerning the cache properties. This research paper showed that the level of cache and access time increases with the absolute hit rate but decreases with the relative hit rate. The number of cache levels varies directly with absolute access time and inversely with relative access time. The level one cache with set associativity of one has the highest access time and as the associativity and cache levels increase the memory access time decreases. The increase in the number of set associativity leads to an increase in cache performance and as well increases the performance speed of a computer.


Introduction
The mismatch between the speed of microprocessor (µP) and main memory (MM) was one of the Bottlenecks that affect the performance of computers [1].Hence, with increasing speed of the microprocessors and the decreasing speed of main memory (as the size of memory increases, the speed of the memory decreases) creates bottleneck when microprocessor accesses data from the memory [2].Main memory is very slow because it is made from Dynamic RAM (DRAM) technology which requires intermittent refreshing.The speed differences between microprocessor and main memory became worst with the introduction of multi-core processors which led to the introduction of many memory hierarchies that takes up some percentages of the total energy consumption [1] [3] [4].Due to the desire for high performance computer systems, cache memory is introduced between the microprocessor and the main memory to reduce the bottleneck.Cache memory is a high speed, smallsized type of volatile computer memory that provides high-speed data access to a microprocessor and stores frequently used computer programs, applications and data until a computer is restarted.Cache memory reduces the access time between the main memory and the microprocessor because it is smaller in size compared to main memory and it is made of Static RAM (SRAM) technology which requires no refreshing.The best cache configuration gives the minimum execution time and the lowest energy consumption.A quality cache configuration encapsulates total cache and block sizes, associativity, search algorithm, pre-fetch and write policies as some of the parameters that make up a good cache configuration [5] [6] [7].
A good configurable cache architecture incorporates three configurable cache parameters, configured by setting a few bits in the configuration register.Cache can be configured in software as direct mapped cache, fully associative cache and N-way set associative cache while utilizing the full capacity of cache.This type of software-based configuration is achieved by employing a technique known as way concatenation [2].Line Concatenation technique is used to configure the cache line size.One of the most architectural configurations used to reduce the energy consumption in cache is portioning of cache into several smaller caches known as levels of caches.These levels of cache also reduce energy consumption, increases efficiency and performance by exploiting locality of reference.Most of the cache energy consumption is due to dynamic power and a small fraction is due to static power.Dynamic power consumption occurs as a result of switching activities of transistors during cache access while static comes from current leakage, even when the cache is not being accessed [8].However, 20-50% of the energy consumption of the on-chip cache is attributed to Translation Lookaside Buffer (TLB).Translation lookaside buffer helps to translate virtual address issued by the processor to physical address present in the cache or the main memory [9] [10] [11].Dynamic energy consumption characterized by first level (L1) caches since they are more frequently accessed than the other levels of cache, due to the parallel access of tag and data arrays.Locality of reference is a rule in computer architecture in which program tends to reuse resources that has been most frequently/recently used [12].The amount of data transferred between the main memory and the write back operation or policy is dependent on the cache line.The larger the cache line size, the more data that is likely related are closed together and brought into the cache block at the same time.When a processor needs a word, it generates a reference address, and it will use the generated reference address to search for the world in cache block.However, if it is in the cache, it will be delivered to the processor and it is known as cache hit, but if it is not in the cache block it will be searched for in main memory and it is known as cache miss.Hit rate is the average percentage or frequent hit in cache by the processor without miss.It was observed in [12] that the memory size is directly proportional to the hit rate, the latency/access time and varies inversely to the miss rate.However, the bigger the memory size, the better the hit rate but the worst the access time.This shows the relationship between the cache levels depending on the designer's choice.The efficiency of high-performance shared memory multiprocessor depends on the design of cache coherence protocol [13] [14] [15].
The expected high performance of future computers depends on the levels of cache memory in the system to effectively curb the delay encountered my microprocessor in fetching instructions and data from memory, multiple cache memory hierarchy are employed.In current microprocessors such as Core i7, there are three levels of cache memory, thus level one (L1) cache, level two (L2) cache and level three (L3) cache.L1 cache is always highly optimized to achieve low latency on memory request hit.Every instruction fetch and every data memory reference rely on the timely response of L1 cache to keep the pipeline structure filled.L1 cache is not normally optimized for low power consumption.However, higher order cache levels (L2, L3 etc.) are optimized for moderate power savings at the cost of slightly prolonged hit latencies and marginally decreased hit rates.Saving of energy in the memory subsystem can effectively control aging effects and also extend lifetime of the cache [5].These compromises in performance are usually not acceptable for L1 caches because of the risk of impacting overall system performance [16] [17] [18] [19].

Literature Review
This model was developed following the normal cache policy principle and memory request flow rate in a multi-level computer memory ( In a multi-level cache system, the resultant hit rate (hr1) and the resultant miss rate (mr1) of L1 are given by ( 2) and (3) respectively. (2) (3) For the L2 cache, the absolute hit rate is h2 and the absolute miss rate (m2) is given by ( 4). (4) In a multi-level cache system, the resultant hit rate (hr2) of the L2 cache is given by ( 5) while the resultant miss rate (mr2) of the L2 cache is given by ( 6).
(5) (6) For the L3 cache, the absolute hit rate is h3 and the absolute miss rate (m3) is given by ( 7).(7) In a multi-level cache system, the resultant hit rate (hr3) of the L3 cache is given by ( 8) while the resultant miss rate (mr3) of L3 cache is given by ( 9).Applying the law of mathematical induction to equations ( 2), ( 3), ( 5), ( 6), ( 8) and ( 9), equations (10) and (11) For a multi-level cache system with cache absolute access time varying from t1 for L1 cache to tn for Ln cache and main memory absolute access time tm, the total access time (tt) required by the microprocessor to retrieve R request from memory and relative access time required by the processor to retrieve some request from cache memory is given by ( 12) and ( 13) respectively.
(12) (13) The cache effectiveness (Ec) of a computer memory system with a multi-level cache system is given by ( 14).(14) The hit rate of cache memory depends on the cache capacity and the associativity of the cache.The hit rate of nth cache memory is given by ( 15).(15) Where A is the associativity of the cache memory and Cn is the capacity of the Ln cache memory in bytes.The capacity of nth cache memory is given by ( 16).(16) Substituting equation (16) in equation (15), equation ( 17) was obtained.(17) The time tn in (14) is the absolute access time of cache memory and it is given by ( 18).(18) 3. Methodology This model was mathematically developed following the acceptable and existing cache policy principle and memory request flow rate in a multi-level computer memory.Finally, the model was developed by applying the law of mathematical induction in equations ( 2), (3), ( 5), ( 6), ( 8) and ( 9) to obtain a universal resultant cache hit rate (h_rn) and resultant cache miss rate (m_rn) for Ln cache.

Finding and Discussion
This section of the paper discussed in detail the results obtained based on the level of cache capacity, associativity, cache absolute hit rate and the performance of cache levels in relation to cache capacity and associativity.
This section of the paper discussed in details the results obtained based on the level of cache capacity, associativity, cache absolute hit rate and the performance of cache levels in relation to cache capacity and associativity.  1, it was observed that associativity varies directly as cache absolute Hit rate and cache capacity in kB.Cache capacity varies directly with the cache absolute Hit rate.

Figure 2. Effects of Cache Capacity and Cache Associativity on Cache Absolute Hit Rate
It was observed from Figure 2 that the higher the cache capacity, the higher the cache hit rate.Furthermore, an increase in associativity and cache capacity leads to an increase in hit rate though decreases speed/increases latency.It was also observed from Figure 2 that from a cache capacity of 1MB the associativity has an infinitesimal /less effect on the hit rate.This implies that the hit rate is dependent on the cache capacity but not solely on associativity.Table 2 shows the relationship between level one cache and the other levels of cache.From equation ( 15), it was observed that as far as the level 1 cache is known every other level can be determined.Table 2 was generated in accordance with equation ( 15) and it concurs with the rule of cache placement policy.From Figure 3, it was observed that as the cache capacity increases progressively, the cache level capacity increases.Increase in the level of cache capacity varies directly with level one cache capacity.The higher-level cache capacity increases with increase in cache capacity.Table 3 shows the relationship between absolute and relative hit rate with cache size and cache level respectively.From the table it was observed that absolute hit rate increases with increase in cache level and relative hit rate decreases with increase in level of cache.From Figure 4, Absolute hit rate varies directly to the cache levels but relative hit rate varies inversely to the cache levels.This implies that as the level of cache increase the access time increases in absolute hit rate but decreases in relative hit rate.From figure 4, it was obviously observed that relative hit rate improves the performance of a computer compare to absolute hit rate because less access time is the expectation of all designers and users of computer.Table 4 shows that, as the cache capacity is increasing the cache absolute access time is increasing with respect to associativity.The access time is also increasing with increase in cache capacity.The associativity increases with increase in cache capacity in relation with cache absolute access time.
Figure 5, shows the variation of absolute and relative cache access time with cache size and cache levels.It was observed that absolute access time varies inversely as the relative access time with respect to number of cache levels.This implies that absolute access time increases with increase in number of cache level but relative access time decreases with increase in number of cache levels.
From Table 5, it was observed that as the number of set associativity increases the total memory access time decreases.Total memory access time varies inversely as the number of cache levels.This implies that a computer with higher number of set associativity and higher number of cache level gives the best performance which is the desire of every computer designer and user.From figure 6, it was observed that L1 cache to L3 cache has a very clear effect on total memory access time with respect to associativity.Furthermore, total memory access time varies inversely as the number of cache levels.From this plot, it was observed that level 1 cache and set associativity of one has the highest access time and as the associativity and cache levels increases the memory access time decreases.This plot shows that to achieve a high-performance computer, a designer should consider higher levels of cache with higher associativity.Table 6 shows the variation of cache effectiveness with cache size and number of cache levels.From the table it is noticed that effectiveness of a cache depends on the set associativity and the cache level.The higher the set associativity and cache levels the more effective the cache.From Figure 7, it was observed that associativity and number of cache levels varies proportionally to the cache effectiveness.It shows that increase in the number of set associativity leads to increase in cache performance.It also implies that as the number of cache level increases cache effectiveness increases.In all increase in set associativity and number of cache levels increase performance of computer.The disadvantage is that it increases the complexity of the system and also cost.

Conclusion
The developed model showed the efficiency rate, relationships and the performance output level of a computer with respect to the cache properties.This research paper showed that the level of cache and access time increases with absolute hit rate but decreases with relative hit rate.The number of cache levels varies directly with absolute access time and inversely with relative access time.The level 1 cache and set associativity of 1 has the highest access time and as the associativity and cache levels increases the memory access time decreases.Finally, this model showed that, relative hit rate improves the performance of a computer, increase in the number of set associativity leads to increase in cache performance and higher levels of cache with higher associativity which also increases the performance of a computer is the earnest desire of all the computer users and designers

Figure 1 .
Figure 1.Memory Request flow in Multi-Level Computer Memory

Figure 3 .
Figure 3. Plot of Effects of L1 Cache Capacity on the Capacity of L2, L3 and L4 Caches

Figure 4 .
Figure 4. Variation of Absolute and Relative Hit Rate with the Cache Level

Figure 5 .
Figure 5. Variation of Absolute Cache Access Time with Cache Capacity and Cache Associativity

Figure 7 .
Figure 7. Variation of Cache Effectiveness with Cache Size and Number of Cache Levels

Table 3 .
Variation of Absolute and Relative Hit Rate with Cache Size and Cache Level Respectively

Table 4 .
Variation of Absolute and Relative Cache Access Time with Cache Size and Cache Level Respectively

Table 6 .
Variation of cache effectiveness with cache size and number of cache levels