Using Triggered Operations to Offload Rendezvous Messages

Historically, MPI implementations have had to choose between eager messaging protocols that require buffering and rendezvous protocols that sacrifice overlap and strong independent progress in some scenarios. The typical choice is to use an eager protocol for short messages and switch to a rendezvous protocol for long messages. If overlap and progress are desired, some implementations offer the option of using a thread. We propose an approach that leverages triggered operations to implement a long message rendezvous protocol that provides strong progress guarantees. The results indicate that a triggered operation based rendezvous can achieve better overlap than a traditional rendezvous implementation and less wasted band width than an eager long protocol.

[1]  Sandia Report,et al.  The Portals 4.0 Message Passing Interface , 2008 .

[2]  Ron Brightwell A Comparison of Three MPI Implementations for Red Storm , 2005, PVM/MPI.

[3]  Amith R. Mamidala,et al.  Lock-Free Asynchronous Rendezvous Design for MPI Point-to-Point Communication , 2008, PVM/MPI.

[4]  Brian Vinter,et al.  A Comparison of Three MPI Implementations , 2004 .

[5]  Jack Dongarra,et al.  Recent Advances in Parallel Virtual Machine and Message Passing Interface, 15th European PVM/MPI Users' Group Meeting, Dublin, Ireland, September 7-10, 2008. Proceedings , 2008, PVM/MPI.

[6]  Sayantan Sur,et al.  RDMA read based rendezvous protocol for MPI over InfiniBand: design alternatives and benefits , 2006, PPoPP '06.

[7]  Dave Olson,et al.  Pathscale InfiniPath: a first look , 2005, 13th Symposium on High Performance Interconnects (HOTI'05).

[8]  Hubertus Franke,et al.  MPI programming environment for IBM SP1/SP2 , 1995, Proceedings of 15th International Conference on Distributed Computing Systems.

[9]  Ahmad Afsahi,et al.  Improving Communication Progress and Overlap in MPI Rendezvous Protocol over RDMA-enabled Interconnects , 2008, 2008 22nd International Symposium on High Performance Computing Systems and Applications.

[10]  Bruce Jacob,et al.  The structural simulation toolkit , 2006, PERV.

[11]  Ahmad Afsahi,et al.  A Speculative and Adaptive MPI Rendezvous Protocol Over RDMA-enabled Interconnects , 2009, International Journal of Parallel Programming.

[12]  Jack Dongarra,et al.  Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Stuttgart, Germany, September 12-15, 2010. Proceedings , 2010, EuroMPI.

[13]  Rolf Riesen,et al.  Design, Implementation, and Performance of MPI on Portals 3.0 , 2003, Int. J. High Perform. Comput. Appl..

[14]  Wu-chun Feng,et al.  The Quadrics Network: High-Performance Clustering Technology , 2002, IEEE Micro.

[15]  Anthony Skjellum,et al.  A High-Performance, Portable Implementation of the MPI Message Passing Interface Standard , 1996, Parallel Comput..

[16]  Karl S. Hemmert,et al.  Using Triggered Operations to Offload Collective Communication Operations , 2010, EuroMPI.

[17]  Keith D. Underwood,et al.  Evaluation of an Eager Protocol Optimization for MPI , 2003, PVM/MPI.