Access Declarations: Revealing Data Access Patterns to Hardware

This work addresses the challenge of programming on Explicitly Managed Memory (EMM) systems. These systems are characterized by a multi-level memory hierarchy where small and fast memories are placed closer to the processor than larger and slower ones. Unlike conventional processors, hardware does not automatically move the data along the memory hierarchy; there is no automatic caching or prefetching. Instead, the onus of managing data placement falls on the software. EMM systems are under active research, and some are already in production, because of the potential to achieve high performance per watt. A key impediment to wide deployment of EMMs is programmability. Having to worry about moving data between the levels of the memory hierarchy is like going back to the pre-virtual-memory days where the programmer had to manually transfer data between the RAM and the hard disk. We argue that the way to overcome this challenge is to use the old idea of separation between the interface and the implementation, but take it to the new level. Our solution introduces a new interface construct, access declarations, that expands the contract between the application and the underlying runtime system. An access declaration augments the abstract data type (ADT) with a meaningful hint about how the application will access the data. Armed with this information, the underlying implementation can pick the optimal memory layout and fetching strategy for the ADT. We demonstrate the effectiveness of this approach on image processing, MapReduce, and graphs applications.