Stream Sockets on SHRIMP

This paper describes an implementation of stream sockets for the SHRIMP multicomputer. SHRIMP supports protected, user-level data transfer, allows user-level code to perform its own buffer management, and separates data transfers from control transfers so that data transfers can be done without the interrupting the receiving node's CPU. Our sockets implementation exploits all of these features to provide high performance. End-to-end latency for 8 byte transfers is 11 microseconds, which is considerably lower than all previous implementations of the sockets interface. For large transfers, we obtain a bandwidth of 13.5 MBytes/sec, which is close to the hardware limit when the receiver must perform a copy. Further experiments with the public-domain benchmarks ttcp and netperf confirm the performance of our implementation.

[1]  Kai Li,et al.  Retrospective: virtual memory mapped network interface for the SHRIMP multicomputer , 1994, ISCA '98.

[2]  Samuel J. Leffler,et al.  The design and implementation of the 4.3 BSD Unix operating system , 1991, Addison-Wesley series in computer science.

[3]  W. Daniel Hillis,et al.  The network architecture of the Connection Machine CM-5 (extended abstract) , 1992, SPAA '92.

[4]  Matthias A. Blumrich.  Network interface for protected, user-level communication , 1996 .

[5]  Armando P. Stettner.  The design and implementation of the 4.3BSD UNIX operating system , 1988 .

[6]  Brian N. Bershad,et al.  Protocol service decomposition for high-performance networking , 1994, SOSP '93.

[7]  A. Saini.  An overview of the Intel Pentium processor , 1993, Digest of Papers. Compcon Spring.

[8]  John Wilkes.  Hamlyn — an interface for sender- based communications , 1992 .

[9]  Seth Copen Goldstein,et al.  Active messages: a mechanism for integrating communication and computation , 1998, ISCA '98.

[10]  Kai Li,et al.  Protected, user-level DMA for the SHRIMP network interface , 1996, Proceedings. Second International Symposium on High-Performance Computer Architecture.

[11]  Charles L. Seitz,et al.  Myrinet: A Gigabit-per-Second Local Area Network , 1995, IEEE Micro.

[12]  Kai Li,et al.  Early Experience with Message-Passing on the SHRIMP Multicomputer , 1996, 23rd Annual International Symposium on Computer Architecture (ISCA'96).

[13]  Dana S. Henry,et al.  A tightly-coupled processor-network interface , 1992, ASPLOS V.

[14]  David E. Culler,et al.  High-performance local area communication with fast sockets , 1997 .

[15]  W. Daniel Hillis,et al.  The Network Architecture of the Connection Machine CM-5 , 1996, J. Parallel Distributed Comput..

[16]  A. Chien,et al.  High Performance Messaging on Workstations: Illinois Fast Messages (FM) for Myrinet , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[17]  Liviu Iftode,et al.  Software support for virtual memory-mapped communication , 1996, Proceedings of International Conference on Parallel Processing.

[18]  H. T. Kung,et al.  Supporting systolic and memory communication in iWarp , 1990, ISCA '90.

[19]  William J. Dally,et al.  Deadlock-Free Message Routing in Multiprocessor Interconnection Networks , 1987, IEEE Transactions on Computers.

[20]  William J. Dally,et al.  The J-machine system , 1991 .

[21]  Kai Li,et al.  Design and implementation of NX message passing using Shrimp virtual memory mapped communication , 1996, Proceedings of the 1996 ICPP Workshop on Challenges for Parallel Processing.

[22]  P. Pierce,et al.  The NX/2 operating system , 1988, C3P.

[23]  Angelos Bilas,et al.  Fast RPC on the SHRIMP Virtual Memory Mapped Network Interface , 1997, J. Parallel Distributed Comput..

[24]  Seth Copen Goldstein,et al.  Active messages: a mechanism for integrating communication and computation , 1998, ISCA '98.

[25]  Thorsten von Eicken,et al.  U-Net: a user-level network interface for parallel and distributed computing , 1995, SOSP.