Mechanisms that separate algorithms from implementations for parallel patterns

Parallel programming implementation details often obfuscate the original algorithm and make later algorithm maintenance difficult. Although parallel programming patterns help guide the structured development of parallel programs, they do not necessarily avoid the code obfuscation problem. In this paper, we observe how emerging and existing programming models realized as programming languages, preprocessor directives, and/or libraries are able to support the Implementation Strategy Patterns that were proposed as part of the Our Pattern Language. We posit that these emerging programming models are in some cases able to avoid code obfuscation through features that prevent tangling of algorithm and implementation details for parallelization and performance optimizations. We qualitatively evaluate these features in terms of how much they prevent tangling and how well they provide programmer-control over implementation details. We conclude with remarks on potential research directions for studying how to produce efficient and maintainable parallel programs by separating algorithm from implementation.

[1]  Katherine Yelick,et al.  UPC: Distributed Shared-Memory Programming , 2003 .

[2]  Bradford L. Chamberlain,et al.  Parallel Programmability and the Chapel Language , 2007, Int. J. High Perform. Comput. Appl..

[3]  Nancy M. Amato,et al.  STAPL: An Adaptive, Generic Parallel C++ Library , 2001, LCPC.

[4]  Cédric Bastoul,et al.  Code generation in the polyhedral model is easier than you think , 2004, Proceedings. 13th International Conference on Parallel Architecture and Compilation Techniques, 2004. PACT 2004..

[5]  Robert W. Numrich,et al.  Co-array Fortran for parallel programming , 1998, FORF.

[6]  Rudolf Eigenmann,et al.  Towards automatic translation of OpenMP to MPI , 2005, ICS '05.

[7]  Katherine A. Yelick,et al.  Titanium: A High-performance Java Dialect , 1998, Concurr. Pract. Exp..

[8]  Nancy M. Amato,et al.  Associative Parallel Containers in STAPL , 2007, LCPC.

[9]  Rice UniversityCORPORATE,et al.  High performance Fortran language specification , 1993 .

[10]  Monica S. Lam,et al.  A data locality optimizing algorithm , 1991, PLDI '91.

[11]  Barbara Liskov,et al.  A history of CLU , 1993, HOPL-II.

[12]  Bradford L. Chamberlain,et al.  The cascade high productivity language , 2004, Ninth International Workshop on High-Level Parallel Programming Models and Supportive Environments, 2004. Proceedings..

[13]  Francisco Jose Arzu Standard Templates Adaptive Parallel Library , 2000 .

[14]  Alexey Kukanov,et al.  The Foundations for Scalable Multicore Software in Intel Threading Building Blocks , 2007 .

[15]  K. Keutzer,et al.  Our Pattern Language ( OPL ) : A Design Pattern Language for Engineering ( Parallel ) Software , 2009 .

[16]  L. Dagum,et al.  OpenMP: an industry standard API for shared-memory programming , 1998 .

[17]  François Irigoin,et al.  Supernode partitioning , 1988, POPL '88.

[18]  Jack Dongarra,et al.  Automatic Blocking of Nested Loops , 1990 .

[19]  Martin Griebl,et al.  Code generation in the polytope model , 1998, Proceedings. 1998 International Conference on Parallel Architectures and Compilation Techniques (Cat. No.98EX192).

[20]  Gregor Kiczales,et al.  Aspect-oriented programming , 2001, ESEC/FSE-9.

[21]  Ami Marowka,et al.  BSP2OMP: A compiler for translating BSP programs to OpenMP , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[22]  M S Waterman,et al.  Identification of common molecular subsequences. , 1981, Journal of molecular biology.

[23]  Laxmikant V. Kalé,et al.  CHARM++: a portable concurrent object oriented system based on C++ , 1993, OOPSLA '93.

[24]  Jin-Soo Kim,et al.  ParADE: An OpenMP Programming Environment for SMP Cluster Systems , 2003, ACM/IEEE SC 2003 Conference (SC'03).

[25]  Nancy M. Amato,et al.  The STAPL pArray , 2007, MEDEA '07.

[26]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[27]  John R. Gilbert,et al.  Aspect-Oriented Programming of Sparse Matrix Code , 1997, ISCOPE.

[28]  Cristina V. Lopes,et al.  Aspect-oriented programming , 1999, ECOOP Workshops.

[29]  Katherine Yelick,et al.  Introduction to UPC and Language Specification , 2000 .