? SC (Or, Can Adding Scalable Locality to Distributed Shared Memory Yield SuperComputer Power?)

Distributed Shared Memory, such as that provided by Intel's Cluster OpenMP, lets program- mers treat the combined memory systems of a cluster of workstations as a single large address space. This relieves the programmer of the burden of explicitly transferring data: a correct OpenMP program should still work with Cluster OpenMP. However, by hiding data trans- fers, such systems also hide a major performance factor: correct OpenMP programs with poor locality-of-reference become correct but intolerably slow Cluster OpenMP programs. Scalable Locality describes the program property of locality that increases with problem size (just as Scalable Parallelism describes the property of parallelism that increases with problem size). In principle, the combination of an optimization that exposes scalable locality and a distributed shared memory system should yield a simple programming model with good performance on a cluster. We have begun to explore a combination of Cluster OpenMP and the Pluto research com- piler's implementation of time tiling, which can produce parallel programs with scalable locality from sequential loop-based dense matrix codes. In this article, we review our approach, discuss our performance model and its implica- tions for tile size selection, and present our most recent experimental tests of the viability of our approach and validity of our performance model. Our performance model captures only machine-independent issues that are critical to setting tile size. It deduces lower bounds on tile dimensions from a combination of purely hardware parameters (e.g. memory bandwidth) and parameters describing the software without reference to any particular hardware (e.g. number of live values produced by the loop nest). We also model load imbalance from OpenMP barriers, which is significant for smaller problems. Our results, while preliminary, are quite encouraging.