Assessing contention effects of all-to-all communications on clusters and grids

One of the most important collective com- munication patterns used in scientific applications is the complete exchange, also called All-to-All. Although ecient algorithms have been studied for specific net- works, general solutions like those available in well- known MPI distributions (e.g. the MPI_Alltoall op- eration) are strongly influenced by the congestion of network resources. In this paper we present an inte- grated approach to model the performance of the All- to-All collective operation, which consists in identifying a contention signature that characterizes a given net- work environment, using it to augment a contention- free communication model. This approach, assessed by experimental results, allows an accurate prediction of the performance of the All-to-All operation over dierent network architectures with a small overhead. We also discuss the problem of network contention in a grid environment, studying some strategies to minimize the impact of contention on the performance of an All- to-All operation.

[1]  Mark J. Clement,et al.  Network Performance Modeling for PVM Clusters , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[2]  Vikram S. Adve,et al.  Analyzing the behavior and performance of parallel programs , 1993 .

[3]  Jesús Labarta,et al.  DiP: A Parallel Program Development Environment , 1996, Euro-Par, Vol. II.

[4]  Jean-Claude König,et al.  Analysis of Gossiping Algorithms with Restricted Buffers , 1998, Parallel Algorithms Appl..

[5]  Michael M. Resch,et al.  Distributed Computing in a Heterogeneous Computing Environment , 1998, PVM/MPI.

[6]  Jack J. Dongarra,et al.  Performance Analysis of MPI Collective Operations , 2005, IPDPS.

[7]  Csaba Andras Moritz,et al.  LoGPC: modeling network contention in message-passing programs , 1998, SIGMETRICS '98/PERFORMANCE '98.

[8]  Kees Verstoep,et al.  Network performance-aware collective communication for clustered wide-area systems , 2001, Parallel Comput..

[9]  S. Lennart Johnsson,et al.  Optimum Broadcasting and Personalized Communication in Hypercubes , 1989, IEEE Trans. Computers.

[10]  Emmanuel Jeannot,et al.  Fast and Efficient Total Exchange on Two Clusters , 2007, Euro-Par.

[11]  T. Tam,et al.  Performance studies of high-speed communication on commodity cluster , 2001 .

[12]  Kenneth R. Jackson,et al.  An Efficient Transposition Algorithm for Distributed Memory Computers , 2002 .

[13]  Edson T. Midorikawa,et al.  PEMPIs: A New Methodology for Modeling and Prediction of MPI Programs Performance , 2004, 16th Symposium on Computer Architecture and High Performance Computing.

[14]  Henri Casanova,et al.  Network modeling issues for grid application scheduling , 2005, Int. J. Found. Comput. Sci..

[15]  Emmanuel Jeannot,et al.  Two fast and efficient message scheduling algorithms for data redistribution through a backbone , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[16]  Hélio Marci de Oliveira,et al.  PEMPIs: A New Methodology for Modeling and Prediction of MPI Programs Performance , 2004, 16th Symposium on Computer Architecture and High Performance Computing.

[17]  Luiz Angelo Steffenel,et al.  Modeling Network Contention Effects on All-to-All Operations , 2006, 2006 IEEE International Conference on Cluster Computing.

[18]  Cho-Li Wang,et al.  Realistic communication model for parallel computing on cluster , 1999, ICWC 99. IEEE Computer Society International Workshop on Cluster Computing.

[19]  Kees Verstoep,et al.  Fast Measurement of LogP Parameters for Message Passing Platforms , 2000, IPDPS Workshops.

[20]  Roger W. Hockney,et al.  The Communication Challenge for MPP: Intel Paragon and Meiko CS-2 , 1994, Parallel Computing.

[21]  Jack J. Dongarra,et al.  Performance analysis of MPI collective operations , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[22]  Csaba Andras Moritz,et al.  LoGPC: Modeling Network Contention in Message-Passing Programs , 2001, IEEE Trans. Parallel Distributed Syst..

[23]  Eli Upfal,et al.  Efficient Algorithms for All-to-All Communications in Multiport Message-Passing Systems , 1997, IEEE Trans. Parallel Distributed Syst..

[24]  Bronis R. de Supinski,et al.  Accurately measuring MPI broadcasts in a computational grid , 1999, Proceedings. The Eighth International Symposium on High Performance Distributed Computing (Cat. No.99TH8469).