A Smart Cache Designed for Embedded Applications

In this paper, we extend our previous investigation of split array and scalar data caches to embedded systems. More specifically we explore reconfigurable data caches where L-1 data caches are optimally partitioned into scalar caches augmented with victim caches and array caches. We do not change cache block size or set-associativities, making it easier to reconfigure cache banks. We also evaluate how any unused portions of cache resources can be used as prefetch buffers and branch target buffers to further improve the performance of applications. Since embedded systems require very careful management of available resources, our approach to configuring L-1 caches can lead to better performance and better energy savings. I. INTRODUCTION For embedded applications, it is necessary to provide the required performance within specified size and power budgets. Studies have found that the on-chip cache is responsible for 50% of the power consumed by an embedded processor (17). Therefore, it is worthwhile investigating new cache organizations to address both performance and power requirements of embedded applications. In this paper we explore how to design reconfigurable caches that achieve high performance for embedded applications while remaining both energy and area efficient. For the last two decades computer architects have proposed various cache-control mechanisms and novel cache architectures that detect program access patterns and fine- tune cache policies to improve both the overall cache use and data localities for desktop applications. Major cache optimization techniques (to improve either or both miss rate and miss penalty) include increasing block size and cache size, increasing associativity, complementing the regular cache with victim cache, prefetching data, including additional cache hierarchies. Since for embedded applications it is necessary to provide the required performance within specified size and power budgets, most of these techniques often are not implemented. In our previous work (20) we have studied each of these different cache-control mechanisms and performed comprehensive evaluation of our proposed partitioned caches. Our results demonstrated that split-caches can outperform all of these conventional cache optimization techniques. In this paper we adapt and further extend these studies for embedded systems, with the primary goal of energy savings while maintaining execution performance, yet using significantly smaller data caches. In addition to partitioning data caches into array (or stream) and scalar caches, we investigate how the split caches can be optimally reconfigured for each application. Our studies show significant savings in power and cache capacities. By using these saved area and power for other architectural features to implement different cache optimization techniques, additional performance gains can be achieved for embedded applications. We assume that caches can be designed to permit reconfigurability (10). Previous studies investigated configuring block sizes and set-associativities. In this paper we only explore configuring caches by changing cache sizes, without changing associativity or block sizes. The reconfigurability is achieved by using a configuration vector that can be loaded with a new configuration before an application starts executing. The optimal cache sizes are explored off-line by searching through possible configurations. Our studies show that for L-1 cache system, reconfigurable caches consisting of an instruction cache with prefetching and split data caches (scalar data cache augmented with victim cache, and a separate array data cache) are effective for embedded systems. With such a L-1 cache organization for embedded applications, our results show significant reductions in the number of cache misses, translating into reduced cache access times; reductions in required cache capacities, power consumptions and reduction in the number of execution cycles. This is primarily because we used separate caches which eliminate conflict among different data type that exhibit divergent access behaviors. Since lower miss rates at L-1 reduce the number of times one needs to access L-2 cache, we can reduce the size of L-2 cache. This saved area can be used for other purposes or further power reductions can be achieved by partially or completely shutting down L-2 caches. The energy savings result from the reduced number of cache misses, which in turn reduces the number of trips to higher levels of memories, often crossing chip boundaries.