New abstractions for data parallel programming

Developing applications is becoming increasingly difficult due to recent growth in machine complexity along many dimensions, especially that of parallelism. We are studying data types that can be used to represent data parallel operations. Developing parallel programs with these data types have numerous advantages and such a strategy should facilitate parallel programming and enable portability across machine classes and machine generations without significant performance degradation. In this paper, we discuss our vision of data parallel programming with powerful abstractions. We first discuss earlier work on data parallel programming and list some of its limitations. Then, we introduce several dimensions along which is possible to develop more powerful data parallel programming abstractions. Finally, we present two simple examples of data parallel programs that make use of operators developed as part of our studies.

[1]  Charles Koelbel.  An overview of High Performance Fortran , 1992, FORF.

[2]  David A. Padua,et al.  Design Issues in Parallel Array Languages for Shared Memory , 2008, SAMOS.

[3]  David A. Padua,et al.  Advanced compiler optimizations for supercomputers , 1986, CACM.

[4]  Naga K. Govindaraju,et al.  GPGPU: general-purpose computation on graphics hardware , 2006, SC.

[5]  W. J. Watson.  The TI ASC: a highly modular and flexible super computer architecture , 1972, AFIPS '72 (Fall, part I).

[6]  Robert E. Millstein,et al.  The ILLIAC IV FORTRAN compiler , 1975, Programming Languages and Compilers for Parallel and Vector Machines.

[7]  Guy L. Steele,et al.  Common Lisp the Language , 1984 .

[8]  David A. Padua,et al.  Programming for parallelism and locality with hierarchically tiled arrays , 2006, PPoPP '06.

[9]  Bradley C. Kuszmaul,et al.  Cilk: an efficient multithreaded runtime system , 1995, PPOPP '95.

[10]  James Reinders,et al.  Intel threading building blocks - outfitting C++ for multi-core processor parallelism , 2007 .

[11]  Basilio B. Fraguela,et al.  Hierarchically Tiled Arrays Vs . Intel Threading Building Blocks for Programming Multicore Systems ? , 2007 .

[12]  David A. Padua,et al.  Programming with tiles , 2008, PPOPP.

[13]  Robert A. van de Geijn,et al.  FLAME: Formal Linear Algebra Methods Environment , 2001, TOMS.

[14]  Guy E. Blelloch,et al.  NESL: A Nested Data-Parallel Language , 1992 .

[15]  Laxmikant V. Kalé,et al.  CHARM++: a portable concurrent object oriented system based on C++ , 1993, OOPSLA '93.

[16]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[17]  Michael Hucka,et al.  Programmer's Manual , 2004 .

[18]  W. Daniel Hillis,et al.  The connection machine , 1985 .

[19]  Jerrold L. Wagener,et al.  Fortran 90 Handbook: Complete Ansi/Iso Reference , 1992 .

[20]  John McCarthy,et al.  LISP 1.5 Programmer's Manual , 1962 .

[21]  Richard M. Brown,et al.  The ILLIAC IV Computer , 1968, IEEE Transactions on Computers.

[22]  Kenneth E. Iverson,et al.  A programming language , 1899, AIEE-IRE '62 (Spring).