Single Node On-Line Simulation of MPI Applications with SMPI

Simulation is a popular approach for predicting the performance of MPI applications for platforms that are not at one's disposal. It is also a way to teach the principles of parallel programming and high-performance computing to students without access to a parallel computer. In this work we present SMPI, a simulator for MPI applications that uses on-line simulation, i.e., the application is executed but part of the execution takes place within a simulation component. SMPI simulations account for network contention in a fast and scalable manner. SMPI also implements an original and validated piece-wise linear model for data transfer times between cluster nodes. Finally SMPI simulations of large-scale applications on large-scale platforms can be executed on a single node thanks to techniques to reduce the simulation's compute time and memory footprint. These contributions are validated via a large set of experiments in which SMPI is compared to popular MPI implementations with a view to assess its accuracy, scalability, and speed.

[1]  A. Lumsdaine,et al.  LogGOPSim: simulating large-scale applications in the LogGOPS model , 2010, HPDC '10.

[2]  Henri Casanova,et al.  SimGrid: A Generic Framework for Large-Scale Distributed Experiments , 2008, Tenth International Conference on Computer Modeling and Simulation (uksim 2008).

[3]  Jesper Larsson Träff,et al.  SKaMPI: a comprehensive benchmark for public benchmarking of MPI , 2002, Sci. Program..

[4]  Andrew A. Chien,et al.  The MicroGrid: a Scientific Tool for Modeling Computational Grids , 2006 .

[5]  Thomas Phan,et al.  Parallel Simulation of Large-Scale Parallel Applications , 2001, Int. J. High Perform. Comput. Appl..

[6]  Alan Wagner,et al.  MPI-NeTSim: A Network Simulation Module for MPI , 2009, 2009 15th International Conference on Parallel and Distributed Systems.

[7]  David M. Nicol,et al.  Parallelized Direct Execution Simulation of Message-Passing Parallel Programs , 1996, IEEE Trans. Parallel Distributed Syst..

[8]  Laxmikant V. Kalé,et al.  BigSim: a parallel simulator for performance prediction of extremely large parallel machines , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[9]  George Bosilca,et al.  Open MPI: Goals, Concept, and Design of a Next Generation MPI Implementation , 2004, PVM/MPI.

[10]  Laxmikant V. Kalé,et al.  Simulation-Based Performance Prediction for Large Parallel Machines , 2005, International Journal of Parallel Programming.

[11]  Henri Casanova,et al.  Speed and accuracy of network simulation in the SimGrid framework , 2007, Valuetools 2007.

[12]  Marc-André Hermanns,et al.  Verifying Causality between Distant Performance Phenomena in Large-Scale MPI Applications , 2009, 2009 17th Euromicro International Conference on Parallel, Distributed and Network-based Processing.

[13]  Jesús Labarta,et al.  A Framework for Performance Modeling and Prediction , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[14]  Rolf Riesen,et al.  Instruction-level simulation of a cluster at scale , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[15]  Laura Carrington,et al.  A Framework for Application Performance Modeling and Prediction , 2002 .

[16]  Jesús Carretero,et al.  New techniques for simulating high performance MPI applications on large storage networks , 2008, 2008 IEEE International Conference on Cluster Computing.

[17]  Arnaud Legrand,et al.  Accuracy study and improvement of network simulation in the SimGrid framework , 2009, SimuTools.

[18]  Wenguang Chen,et al.  PHANTOM: predicting performance of parallel applications on large-scale parallel machines using a single node , 2010, PPoPP '10.

[19]  Emmanuel Jeannot,et al.  Defining and controlling the heterogeneity of a cluster: The Wrekavoc tool , 2010, J. Syst. Softw..

[20]  Michael Laurenzano,et al.  PSINS: An Open Source Event Tracer and Execution Simulator , 2009, 2009 DoD High Performance Computing Modernization Program Users Group Conference.

[21]  Christian Scheideler,et al.  SIMLAB-a simulation environment for storage area networks , 2001, Proceedings Ninth Euromicro Workshop on Parallel and Distributed Processing.

[22]  Rizos Sakellariou,et al.  Compiler-Optimized Simulation of Large-Scale Applications on High Performance Architectures , 2002, J. Parallel Distributed Comput..

[23]  Rolf Riesen.  A Hybrid MPI Simulator , 2006, 2006 IEEE International Conference on Cluster Computing.

[24]  William Gropp,et al.  MPICH2: A New Start for MPI Implementations , 2002, PVM/MPI.

[25]  Duncan A. Grove,et al.  Communication Benchmarking and Performance Modelling of MPI Programs on Cluster Computers , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[26]  Ewa Deelman,et al.  Asynchronous Parallel Simulation of Parallel Programs , 2000, IEEE Trans. Software Eng..