A Review of Parallelization Tools and Introduction to Easypar

Multicore processors have paved the way to increase the performance of any application by the virtue of benefits of parallelization. However, exploiting parallelism from a program is not easy, as it requires parallel programming expertise. In addition, manual parallelization is a cumbersome, time consuming and inefficient process. A number of tools proposed in the past ease the effort of parallel programming. This paper presents a classification of such parallelization tools. The classification is based on different eras of tool development, role playedby these tools in various parallelization stages, and features provided by parallel program assistance tools. Classification of tools concludes with a discussion on requirements of futuristic parallelization tools. Finally, this paper proposesour on-going work about the development of a parallel program assistance tool called EasyPar, which is a parallel program assistance tool.

[1]  Barton P. Miller,et al.  Incremental call‐path profiling , 2007, Concurr. Comput. Pract. Exp..

[2]  Barbara G. Ryder,et al.  Incremental data-flow analysis algorithms , 1988, TOPL.

[3]  Manoj Kumar,et al.  Measuring Parallelism in Computation-Intensive Scientific/Engineering Applications , 1988, IEEE Trans. Computers.

[4]  Muhammed Al-Mulhem.  Concurrent programming in VISO , 2000 .

[5]  Jesús Labarta,et al.  Interfacing Computer Aided Parallelization and Performance Analysis , 2003, International Conference on Computational Science.

[6]  Yuxiong He,et al.  The Cilkview scalability analyzer , 2010, SPAA '10.

[7]  Tim A. Wagner,et al.  Efficient and flexible incremental parsing , 1998, TOPL.

[8]  Uday Bondhugula,et al.  A practical automatic polyhedral parallelizer and locality optimizer , 2008, PLDI '08.

[9]  V. Sarkar,et al.  Automatic partitioning of a program dependence graph into parallel tasks , 1991, IBM J. Res. Dev..

[10]  Koen De Bosschere,et al.  RecPlay: a fully integrated practical record/replay system , 1999, TOCS.

[11]  Kevin P. McAuliffe,et al.  An architecture for a scientific visualization system , 1992, Proceedings Visualization '92.

[12]  James Reinders,et al.  Intel® threading building blocks , 2008 .

[13]  Vinay G. Vaidya,et al.  Automatic Sequential to Parallel Code Conversion The S2P Tool and Performance Analysis , 2010 .

[14]  Utpal Banerjee,et al.  Dependence analysis for supercomputing , 1988, The Kluwer international series in engineering and computer science.

[15]  Konstantin Serebryany,et al.  ThreadSanitizer: data race detection in practice , 2009, WBIA '09.

[16]  Martin C. Rinard,et al.  Automatic parallelization of divide and conquer algorithms , 1999, PPoPP '99.

[17]  Saturnino Garcia,et al.  Kismet: parallel speedup estimates for serial programs , 2011, OOPSLA '11.

[18]  Keshav Pingali,et al.  Optimistic parallelism requires abstractions , 2007, PLDI '07.

[19]  Kathryn S. McKinley,et al.  Automatic and interactive parallelization , 1992 .

[20]  Xiangyu Zhang,et al.  Alchemist: A Transparent Dependence Distance Profiling Infrastructure , 2009, 2009 International Symposium on Code Generation and Optimization.

[21]  M. C. Pong.  I-Pigs: An Interactive Graphical Environment for Concurrent Programming , 1991, Comput. J..

[22]  Eitan Farchi,et al.  Multicore Software Engineering, Performance, and Tools , 2013, Lecture Notes in Computer Science.

[23]  Usman Ismail.  Incremental Call Graph Construction for the Eclipse IDE , 2009 .

[24]  Nir Shavit,et al.  Software transactional memory , 1995, PODC '95.

[25]  Monica S. Lam,et al.  Jade: a high-level, machine-independent language for parallel programming , 1993, Computer.

[26]  G. Amdhal,et al.  Validity of the single processor approach to achieving large scale computing capabilities , 1967, AFIPS '67 (Spring).

[27]  Saturnino Garcia,et al.  Bridging the Parallelization Gap : Automating Parallelism Discovery and Planning , 2010 .

[28]  Yuanyuan Zhou,et al.  CTrigger: exposing atomicity violation bugs from their hiding places , 2009, ASPLOS.

[29]  David H. Laidlaw,et al.  The application visualization system: a computational environment for scientific visualization , 1989, IEEE Computer Graphics and Applications.

[30]  Jack J. Dongarra,et al.  On Using Incremental Profiling for the Performance Analysis of Shared Memory Parallel Applications , 2007, Euro-Par.

[31]  Milind Girkar,et al.  Automatic Extraction of Functional Parallelism from Ordinary Programs , 1992, IEEE Trans. Parallel Distributed Syst..

[32]  Eileen Kraemer,et al.  Toward flexible control of the temporal mapping from concurrent program events to animations , 1994, Proceedings of 8th International Parallel Processing Symposium.

[33]  Thomas E. Anderson,et al.  Quartz: a tool for tuning parallel program performance , 1990, SIGMETRICS '90.

[34]  Zhiyuan Li,et al.  An Efficient Data Dependence Analysis for Parallelizing Compilers , 1990, IEEE Trans. Parallel Distributed Syst..

[35]  Monica S. Lam,et al.  Maximizing Multiprocessor Performance with the SUIF Compiler , 1996, Digit. Tech. J..

[36]  Barbara G. Ryder,et al.  An efficient hybrid algorithm for incremental data flow analysis , 1989, POPL '90.

[37]  Antonia Zhai,et al.  A scalable approach to thread-level speculation , 2000, Proceedings of 27th International Symposium on Computer Architecture (IEEE Cat. No.RS00201).

[38]  K. Sarath Kumar,et al.  Hybrid Approach for Parallelization of Sequential Code with Function Level and Block Level Parallelization , 2006, International Symposium on Parallel Computing in Electrical Engineering (PARELEC'06).

[39]  Barbara G. Ryder.  A position paper on compile-time program analysis , 1997, SIGP.

[40]  Yunheung Paek,et al.  Parallel Programming with Polaris , 1996, Computer.

[41]  Brian Q. Brode,et al.  DEEP: a development environment for parallel programs , 1998, Proceedings of the First Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing.

[42]  W. Ambrus.  A framework for automatic parallelization of sequential programs , 2003, Proceedings of the 7th International Conference on Telecommunications, 2003. ConTEL 2003..

[43]  Nathan R. Tallent,et al.  Effective performance measurement and analysis of multithreaded applications , 2009, PPoPP '09.

[44]  Eileen Kraemer,et al.  Applying Program Visualization Techniques to Aid Parallel and Distributed Program Development , 1991 .

[45]  Antti Virtanen,et al.  Convit, a Tool for Learning Concurrent Programming , 2003 .

[46]  M. S. Krishnamoorthy,et al.  An object-based parallel programming assistant , 1988, OOPSLA/ECOOP '88.

[47]  Rudolf Eigenmann,et al.  Cetus: A Source-to-Source Compiler Infrastructure for Multicores , 2009, Computer.

[48]  Keir Fraser,et al.  Language support for lightweight transactions , 2003, SIGP.

[49]  Russ Abbott.  An Integrated Approach to Software Development , 1986 .

[50]  Chen Ding,et al.  Software behavior oriented parallelization , 2007, PLDI '07.

[51]  Per Stenström.  The Paradigm Shift to Multi-Cores: Opportunities and Challenges , 2007 .

[52]  Koen De Bosschere,et al.  Extracting coarse-grain parallelism in general-purpose programs , 2008, PPOPP.

[53]  Chau-Wen Tseng,et al.  An Overview of the SUIF Compiler for Scalable Parallel Machines , 1995, PPSC.

[54]  Michael F. P. O'Boyle,et al.  Towards a holistic approach to auto-parallelization: integrating profile-driven parallelism detection and machine-learning based mapping , 2009, PLDI '09.

[55]  Péter Kacsuk,et al.  The GRED graphical editor for the GRADE parallel program development environment , 1998, Future Gener. Comput. Syst..

[56]  Björn Franke,et al.  Semi-automatic extraction and exploitation of hierarchical pipeline parallelism using profiling information , 2010, 2010 19th International Conference on Parallel Architectures and Compilation Techniques (PACT).

[57]  Ken Kennedy,et al.  Efficient computation of flow insensitive interprocedural summary information , 1984, SIGPLAN '84.

[58]  Manish Gupta,et al.  Automatic Parallelization of Recursive Procedures , 1999, 1999 International Conference on Parallel Architectures and Compilation Techniques (Cat. No.PR00425).

[59]  Monica S. Lam,et al.  Interprocedural Analysis for Parallelization , 1995, LCPC.

[60]  Ken Kennedy,et al.  Fast interprocedual alias analysis , 1989, POPL '89.

[61]  Minjang Kim.  Dynamic program analysis algorithms to assist parallelization , 2012 .

[62]  Barbara G. Ryder,et al.  Incremental Analysis of MOD Problem for C , 1995 .

[63]  Jeff Magee,et al.  Graphical configuration programming , 1989, Computer.

[64]  Dirk Grunwald,et al.  Identifying potential parallelism via loop-centric profiling , 2007, CF '07.

[65]  Leon Moonen,et al.  On the Use of Data Flow Analysis in Static Profiling , 2008, 2008 Eighth IEEE International Working Conference on Source Code Analysis and Manipulation.

[66]  Marco Furini,et al.  International Journal of Computer and Applications , 2010 .

[67]  Peng Wu,et al.  Experiences of using a dependence profiler to assist parallelization for multi-cores , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW).

[68]  Jeffrey H. Meyerson,et al.  The Go Programming Language , 2014, IEEE Softw..

[69]  Ebba Þóra Hvannberg,et al.  An object-based parallel programming assistant , 1989, ACM SIGPLAN Notices.

[70]  Keshav Pingali,et al.  How much parallelism is there in irregular applications? , 2009, PPoPP '09.

[71]  Hyesoon Kim,et al.  SD3: A Scalable Approach to Dynamic Data-Dependence Profiling , 2010, 2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture.