Data layout optimization techniques for modern and emerging architectures

The never-ending pursuit of higher performance is one fundamental driving force of computer science research. Although the semiconductor industry has fulfilled Moore’s Law over the last forty years by doubling transistor density every two years, the effectiveness of hardware advances cannot be fully exploited due to the mismatch between the architectural environment and the user program. Program optimization is a key to bridge this gap. In this dissertation, instead of restructuring programs’ control flow as in many previous efforts, we have applied several new data layout optimization techniques to answer many optimization challenges on modern and emerging architectures. In particular, the developed techniques and their unique contributions are as follows. (1) We describe an approach where a class of computations is modeled in terms of constituent operations that are empirically measured, thereby allowing modeling of the overall execution time. The performance model with empirically determined cost components is used to perform data layout optimization in the context of the Tensor Contraction Engine, a compiler for a high-level domain-specific language for expressing computational models in quantum chemistry. (2) To obtain a highly optimized index permutation library for dynamic data layout optimization, we develop an integrated optimization framework that addresses a number of issues including tiling for the memory hierarchy, effective handling of memory misalignment, utilizing memory subsystem characteristics, and the exploitation of the parallelism provided by the vector instruction sets in current processors. A judicious combination of analytical and empirical approaches is used to determine the most appropriate optimizations. (3) With increasing numbers of cores, future CMPs (Chip Multi-Processors) are likely to have a tiled architecture with a portion of shared L2 cache on each tile and a bank-interleaved distribution of the address space. Although such an organization is effective for avoiding access hot-spots, it can cause a significant number of non-local L2 accesses for many commonly occurring regular data access patterns. We develop a compile-time framework for data locality optimization via data layout transformation. Using a polyhedral model, the program’s localizability is determined by analysis of its index set and array reference functions, followed by non-canonical data layout transformation to reduce non-local accesses for localizable computations. (4) We leverage software and operating system utilities to identify locality patterns of data objects and allocate them accordingly with different priorities in caches. This data object locality guided caching strategy is mainly designed to address the inability of LRU replacement to effectively handle memory intensive programs with weak locality (such as streaming accesses) and contention among strong locality data objects in caches, so that sub-optimal replacement decisions can be avoided. To achieve our goal, we present a system software framework. We first collect object-relative reuse distance histograms and inter-object interference histograms via memory trace sampling. With several low-cost training runs, we are able to determine the locality patterns of data objects. For the actual runs, we categorize data objects into different locality types and partition the cache space among data objects with a heuristic algorithm, in order to reduce cache misses through segregation of contending objects. The object-level cache partitioning framework has been implemented through modification of a Linux kernel.