To INT_MAX... and Beyond! Exploring Large-Count Support in MPI

In order to describe a structured region of memory, the routines in the MPI standard use a (count, datatype) pair. The C specification for this convention uses an int type for the count. Since C int types are nearly always 32 bits large and signed, counting more than 231 elements poses a challenge. Instead of changing the existing MPI routines, and all consumers of those routines, the MPI Forum asserts that users can build up large datatypes from smaller types. To evaluate this hypothesis and to provide a user-friendly solution to the large-count issue, we have developed BigMPI, a library on top of MPI that maps large-count MPI-like functions to MPI-3 standard features. BigMPI demonstrates a way to perform such a construction, reveals shortcomings of the MPI standard, and uncovers bugs in MPI implementations.

[1]  Ieee Standards Board.  System application program interface (API) (C language) , 1990 .

[2]  Robert J. Harrison,et al.  Global Arrays: a portable "shared-memory" programming model for distributed memory computers , 1994, Proceedings of Supercomputing '94.

[3]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[4]  William Gropp,et al.  Efficient Management of Parallelism in Object-Oriented Numerical Software Libraries , 1997, SciTools.

[5]  Alex Rapaport,et al.  Mpi-2: extensions to the message-passing interface , 1997 .

[6]  Rajeev Thakur,et al.  On implementing MPI-IO portably and with high performance , 1999, IOPADS '99.

[7]  Bryan Carpenter,et al.  ARMCI: A Portable Remote Memory Copy Libray for Ditributed Array Libraries and Compiler Run-Time Systems , 1999, IPPS/SPDP Workshops.

[8]  G. Allen,et al.  Supporting Efficient Execution in Heterogeneous Distributed Computing Environments with Cactus and Globus , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[9]  Ian T. Foster,et al.  Supporting Efficient Execution in Heterogeneous Distributed Computing Environments with Cactus and Globus , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[10]  Jianwei Li,et al.  Parallel netCDF: A High-Performance Scientific I/O Interface , 2003, ACM/IEEE SC 2003 Conference (SC'03).

[11]  Robert Latham,et al.  Extending the MPI-2 Generalized Request Interface , 2007, PVM/MPI.

[12]  Carsten Kutzner,et al.  GROMACS 4:  Algorithms for Highly Efficient, Load-Balanced, and Scalable Molecular Simulation. , 2008, Journal of chemical theory and computation.

[13]  Dietmar Fey,et al.  LibGeoDecomp: A Grid-Enabled Library for Geometric Decomposition Codes , 2008, PVM/MPI.

[14]  John Shalf,et al.  Exascale Computing Technology Challenges , 2010, VECPAR.

[15]  Satoshi Matsuoka,et al.  Physis: An implicitly parallel programming model for stencil computations on large-scale GPU-accelerated supercomputers , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[16]  Torsten Hoefler,et al.  Mpi on Millions of Cores * , 2022 .

[17]  Nagiza F. Samatova,et al.  Enabling Fast, Noncontiguous GPU Data Movement in Hybrid MPI+GPU Environments , 2012, 2012 IEEE International Conference on Cluster Computing.

[18]  Jeffery A Kuehn,et al.  OpenSHMEM Application Programming Interface, v1.0 Final , 2012 .

[19]  Barbara M. Chapman,et al.  Implementing OpenSHMEM Using MPI-3 One-Sided Communication , 2014, OpenSHMEM.