MPI at Exascale

With petascale systems already available, researchers are devoting their attention to the issues needed to reach the next major level in performance, namely, exascale. Explicit message passing using the Message Passing Interface (MPI) is the most commonly used model for programming petascale systems today. In this paper, we investigate what is needed to enable MPI to scale to exascale, both in the MPI specification and in MPI implementations, focusing on issues such as memory consumption and performance. We also present results of experiments related to MPI memory consumption at scale on the IBM Blue Gene/P at Argonne National Laboratory.

[1]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[2]  Robert W. Numrich,et al.  Co-array Fortran for parallel programming , 1998, FORF.

[3]  Jack J. Dongarra,et al.  FT-MPI: Fault Tolerant MPI, Supporting Dynamic Applications in a Dynamic World , 2000, PVM/MPI.

[4]  Thomas Hérault,et al.  MPICH-V: Toward a Scalable Fault Tolerant MPI for Volatile Nodes , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[5]  Katherine Yelick,et al.  UPC: Distributed Shared-Memory Programming , 2003 .

[6]  Haoqiang Jin,et al.  Comparing the OpenMP, MPI, and Hybrid Programming Paradigm on an SMP Cluster , 2003 .

[7]  Jesper Larsson Träff.  SMP-Aware Message Passing Programming , 2003, IPDPS.

[8]  Jesper Larsson Träff.  More efficient Reduction Algorithms for Message-Passing Parallel Systems , 2004 .

[9]  Jesper Larsson Träff.  A Simple Work-Optimal Broadcast Algorithm for Message-Passing Parallel Systems , 2004, PVM/MPI.

[10]  Jason Duell,et al.  Problems with using MPI 1.1 and 2.0 as compilation targets for parallel language implementations , 2004, Int. J. High Perform. Comput. Netw..

[11]  Rajeev Thakur,et al.  Optimization of Collective Communication Operations in MPICH , 2005, Int. J. High Perform. Comput. Appl..

[12]  Rajeev Thakur,et al.  Thread-safety in an MPI implementation: Requirements and analysis , 2007, Parallel Comput..

[13]  Robert A. van de Geijn,et al.  Collective communication: theory, practice, and experience , 2007, Concurr. Comput. Pract. Exp..

[14]  Satoshi Matsuoka,et al.  ABARIS: An Adaptable Fault Detection/Recovery Component Framework for MPIs , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[15]  Ewing L. Lusk,et al.  Early Experiments with the OpenMP/MPI Hybrid Programming Model , 2008, IWOMP.

[16]  Edgar Gabriel,et al.  Evaluating Sparse Data Storage Techniques for MPI Groups and Communicators , 2008, ICCS.

[17]  Rajeev Thakur,et al.  Toward message passing for a million processes: characterizing MPI on a massive scale blue gene/P , 2009, Computer Science - Research and Development.

[18]  Rajeev Thakur,et al.  Test suite for evaluating performance of multithreaded MPI communication , 2009, Parallel Comput..

[19]  Torsten Hoefler,et al.  Sparse collective operations for MPI , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[20]  Ewing Lusk,et al.  Fault Tolerance in MPI Programs , 2009 .

[21]  A. Rane,et al.  Experiences in Tuning Performance of Hybrid MPI / OpenMP Applications on Quad-core Systems , 2009 .

[22]  Georg Hager,et al.  Hybrid MPI/OpenMP Parallel Programming on Clusters of Multi-Core SMP Nodes , 2009, 2009 17th Euromicro International Conference on Parallel, Distributed and Network-based Processing.

[23]  Alan Wagner,et al.  Scalability of communicators and groups in MPI , 2010, HPDC '10.

[24]  Rajeev Thakur,et al.  Hybrid parallel programming with MPI and unified parallel C , 2010, Conf. Computing Frontiers.

[25]  Jesper Larsson Träff,et al.  Self-Consistent MPI Performance Guidelines , 2010, IEEE Transactions on Parallel and Distributed Systems.

[26]  Jesper Larsson Träff.  Compact and Efficient Implementation of the MPI Group Operations , 2010, EuroMPI.

[27]  Hubert Ritzdorf,et al.  The scalable process topology interface of MPI 2.2 , 2011, Concurr. Comput. Pract. Exp..