Teaching old caches new tricks: RegionTracker and predictor virtualization

On-chip last-level caches are increasing to tens of megabytes to accommodate applications with large memory footprints and to compensate for high memory latencies and limited off-chip bandwidth. This paper reviews two on-going research efforts that exploit such large caches: coarse-grain cache management, and predictor virtualization. Coarse-grain cache management collects and stores cache information at a large memory region granularity (e.g., 1KB to 8KB). This coarse view of memory access behaviour enables optimizations that were not previously possible with conventional caches. Predictor virtualization is motivated by the observation that on-chip storage has become sufficiently large to accommodate allocating, on demand, a small percentage of its capacity for purposes other than storing program data and instructions. Predictor virtualization uses conventional caches to store program metadata, i.e., information about program behaviour. Such metadata information can be used for several optimizations that improve performance and power. This paper summarizes the progress made and the on-going activity in these two research efforts.