VMMC-2 : Efficient Support for Reliable, Connection-Oriented Communication

The basic virtual memory-mapped communication (VMMC) model provides protected, direct communication between the sender's and receiver's virtual address spaces, but it does not support high-level connection-oriented communication APIs well. This paper presents VMMC-2, an extension to the basic VMMC. We describe the design, implementation, and evaluate the performance of three mechanisms in VMMC-2: (1) a user-managed TLB mechanism for address translation which enables user libraries to dynamically manage the amount of pinned space and requires only driver support from many operating systems; (2) a transfer redirection mechanism which avoids copying on the receiver's side; (3) a reliable communication protocol at the data link layer which avoids copying on the sender's side. To validate our extensions we implemented stream sockets on top of the VMMC-2 running on a Myrinet network of Pentium PCs. This zero-copy sockets implementation provides a maximum bandwidth of over 84 Mbytes/s and a one-way latency of 20 s.

[1]  Thu D. Nguyen,et al.  Implementing Network Protocols at User Level , 1993, SIGCOMM.

[2]  Paul J. Leach,et al.  The Architecture of an Integrated Local Network , 1983, IEEE J. Sel. Areas Commun..

[3]  Kai Li,et al.  Design and implementation of NX message passing using Shrimp virtual memory mapped communication , 1996, Proceedings of the 1996 ICPP Workshop on Challenges for Parallel Processing.

[4]  Armando P. Stettner.  The design and implementation of the 4.3BSD UNIX operating system , 1988 .

[5]  Michael C. Browne,et al.  S-Connect: from networks of workstations to supercomputer performance , 1995, Proceedings 22nd Annual International Symposium on Computer Architecture.

[6]  Joseph Pasquale,et al.  The importance of non-data touching processing overheads in TCP/IP , 1993, SIGCOMM 1993.

[7]  Richard P. Martin,et al.  LogP Performance Assessment of Fast Network Interfaces , 1995 .

[8]  Alfred Z. Spector,et al.  Performing remote operations efficiently on a local computer network , 1981, SOSP.

[9]  Jo Asplin,et al.  On the design and performance of the PARFUM Parallel Fault Tolerant Volume Renderer , 1997 .

[10]  Angelos Bilas,et al.  Fast RPC on the SHRIMP Virtual Memory Mapped Network Interface , 1997, J. Parallel Distributed Comput..

[11]  Larry L. Peterson,et al.  Experiences with a high-speed network adaptor: a software perspective , 1994 .

[12]  Larry L. Peterson,et al.  Fbufs: a high-bandwidth cross-domain transfer facility , 1994, SOSP '93.

[13]  J.M. Smith,et al.  Giving applications access to Gb/s networking , 1993, IEEE Network.

[14]  Angelos Bilas,et al.  Client-server computing on Shrimp , 1997, IEEE Micro.

[15]  Andrew A. ChienJanuary.  Fast Messages ( FM ) : E cient , Portable Communication for Workstation Clusters and Massively-Parallel Processors , 1997 .

[16]  Seth Copen Goldstein,et al.  Active messages: a mechanism for integrating communication and computation , 1998, ISCA '98.

[17]  Kai Li,et al.  Retrospective: virtual memory mapped network interface for the SHRIMP multicomputer , 1994, ISCA '98.

[18]  Willy Zwaenepoel,et al.  The peregrine high‐performance RPC system , 1993, Softw. Pract. Exp..

[19]  P. Pierce,et al.  The Paragon implementation of the NX message passing interface , 1994, Proceedings of IEEE Scalable High Performance Computing Conference.

[20]  David E. Culler,et al.  High-performance local area communication with fast sockets , 1997 .

[21]  Brian Zill,et al.  Protocol implementation on the Nectar Communication Processor , 1990, SIGCOMM '90.

[22]  Joseph Pasquale,et al.  The importance of non-data touching processing overheads in TCP/IP , 1993, SIGCOMM '93.

[23]  Milon Mackey,et al.  An implementation of the Hamlyn sender-managed interface architecture , 1996, OSDI '96.

[24]  Liviu Iftode,et al.  Improving release-consistent shared virtual memory using automatic update , 1996, Proceedings. Second International Symposium on High-Performance Computer Architecture.

[25]  J. Lumley,et al.  A high-throughput network interface to a RISC workstation , 1992, IEEE Workshop on the Architecture and Implementation of High Performance Communication Subsystems.

[26]  Thorsten von Eicken,et al.  Incorporating Memory Management into User-Level Network Interfaces , 1997 .

[27]  Kai Li,et al.  Design and implementation of virtual memory-mapped communication on Myrinet , 1997, Proceedings 11th International Parallel Processing Symposium.

[28]  Seth Copen Goldstein,et al.  Active messages: a mechanism for integrating communication and computation , 1998, ISCA '98.

[29]  Thorsten von Eicken,et al.  U-Net: a user-level network interface for parallel and distributed computing , 1995, SOSP.

[30]  Anna R. Karlin,et al.  Competitive randomized algorithms for non-uniform problems , 1990, SODA '90.

[31]  Liviu Iftode,et al.  Software support for virtual memory-mapped communication , 1996, Proceedings of International Conference on Parallel Processing.