MPICH-PM: Design and Implementation of Zero Copy MPI for PM

This report describes the design and implementation of a high performance MPI library using a zero copy message transfer primitive supported by PM. MPICH-PM consists of the MPICH implementation of the Message Passing Interface (MPI) standard, ported to the high-performance, communications library PM. The zero copy message transfer primitive requires a memory area pinned down to physical memory, which is a restricted resource quantity under a paging memory system. Allocation of pinned down memory by multiple simultaneous requests for sending and receiving without any control can cause deadlock. To avoid this deadlock, we have introduced: i) separate of control of send/receive pin-down memory areas to ensure that at least one send and receive may be concurrently processed, and ii) delayed queues to handle the postponed message passing operations which could not be pinned-down.