A Data Cache with Multiple Caching Strategies Tuned to Different Types of Locality

Current data cache organizations fail to deliver high performance in scalar processors for many vector applications. There are two main reasons for this loss of performance: the use of the same organization for caching both spatial and temporal locality and the “eager” caching policy used by caches. The first issue has led to the well-known trade-off of designing caches with a line size of a few tens of bytes. However, for memory reference patterns with low spatial locality a significant pollution is introduced. On the other hand, when the spatial locality is very high, larger lines could be more convenient. The eager caching policy refers to the fact that data that miss in the cache and is required by the processor is always cached (excepting writes in a no write allocate cache). However, it is common in numerical applications to have large working sets (large vectors, larger than the cache size), that result on a swept of the cache without any opportunity to exploit temporal locality. In addition, they replace some other data that may be required later. In this paper, a novel data cache organization, called dual data cache, is presented. To our knowledge, this is the first time a cache with independent parts for managing spatial and temporal locality 1s proposed. In this way, both types of locality can be more efficiently exploited. In addition, the dual data cache implements a lazy caching policy, which tr]es not to cache something until a benefit (in terms of spatial or temporal locality) can be predicted. For both purposes, the dual data cache makes use of a locality prediction table, which is a history table with information about the most recently executed Ioad/store instructions. In addition, a simplified implementation of the dual data cache, which is called selective cache is also presented.