Investigating Scenario-Conscious Asynchronous Rendezvous over RDMA

In this paper, we propose a light-weight asynchronous message progression mechanism for large message transfers in Message Passing Interface (MPI) Rendezvous protocol that is scenario-conscious and consequently overhead-free in cases where independent message progression naturally happens. Without requiring a dedicated thread, we take advantage of small bursts of CPU to poll for message transfer conditions. The existing application thread is parasitized for the purpose of getting those small bursts of CPU. Our proposed approach is only triggered when the message transfer would otherwise be deferred to the MPI wait call, and it allows for full message progression, achieving 100% overlap. It does not add to the memory footprint of the applications, and is effective in improving the communication performance of most of the applications studied in this paper.

[1]  Amith R. Mamidala,et al.  Lock-Free Asynchronous Rendezvous Design for MPI Point-to-Point Communication , 2008, PVM/MPI.

[2]  A. Dickinson.  CMOS Photonics - Bringing Moore's Law to Optical Interconnect , 2007 .

[3]  Amy W. Apon,et al.  Implementation and design analysis of a network messaging module using virtual interface architecture , 2004, 2004 IEEE International Conference on Cluster Computing (IEEE Cat. No.04EX935).

[4]  Ahmad Afsahi,et al.  Assessing the Ability of Computation/Communication Overlap and Communication Progress in Modern Interconnects , 2007, 15th Annual IEEE Symposium on High-Performance Interconnects (HOTI 2007).

[5]  Scott Pakin.  Receiver-initiated message passing over RDMA Networks , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[6]  Xin Yuan,et al.  Near-Optimal Rendezvous Protocols for RDMA-Enabled Clusters , 2010, 2010 39th International Conference on Parallel Processing.

[7]  Ahmad Afsahi,et al.  A Speculative and Adaptive MPI Rendezvous Protocol Over RDMA-enabled Interconnects , 2009, International Journal of Parallel Programming.

[8]  David H. Bailey,et al.  The Nas Parallel Benchmarks , 1991, Int. J. High Perform. Comput. Appl..

[9]  Sayantan Sur,et al.  RDMA read based rendezvous protocol for MPI over InfiniBand: design alternatives and benefits , 2006, PPoPP '06.

[10]  Xin Yuan,et al.  Maximizing MPI point-to-point communication performance on RDMA-enabled clusters with customized protocols , 2009, ICS.

[11]  Torsten Hoefler,et al.  Message progression in parallel computing - to thread or not to thread? , 2008, 2008 IEEE International Conference on Cluster Computing.