Cut-through delivery in Trapeze: An exercise in low-latency messaging

New network technology continues to improve both the latency and bandwidth of communication in computer clusters. The fastest high-speed networks approach or exceed the I/O bus bandwidths of "gigabit-ready" hosts. These advances introduce new considerations for the design of network interfaces and messaging systems for low-latency communication. This paper investigates cut-through delivery, a technique for overlapping host I/O DMA transfers with network traversal. Cut-through delivery significantly reduces end-to-end latency of large messages, which are often critical for application performance. We have implemented cut-through delivery in Trapeze, a new messaging substrate for network memory and other distributed operating system services. Our current Trapeze prototype is capable of demand-fetching 8 K virtual memory pages in 200 /spl mu/s across a Myrinet cluster of DEC AlphaStations.

[1]  Jack Dongarra,et al.  Pvm 3 user's guide and reference manual , 1993 .

[2]  Milon Mackey,et al.  An implementation of the Hamlyn sender-managed interface architecture , 1996, OSDI '96.

[3]  Anna R. Karlin,et al.  Implementing global memory management in a workstation cluster , 1995, SOSP.

[4]  Andrew A. ChienJanuary.  Fast Messages ( FM ) : E cient , Portable Communication for Workstation Clusters and Massively-Parallel Processors , 1997 .

[5]  Thorsten von Eicken,et al.  U-Net: a user-level network interface for parallel and distributed computing , 1995, SOSP.

[6]  Peter Druschel,et al.  Experiences with a high-speed network adaptor: a software perspective , 1994, SIGCOMM 1994.

[7]  Rolf Hempel,et al.  The MPI Message Passing Interface Standard , 1994 .

[8]  Guru M. Parulkar,et al.  The APIC approach to high performance network interface design: protected DMA and other techniques , 1997, Proceedings of INFOCOM '97.

[9]  Seth Copen Goldstein,et al.  Active Messages: A Mechanism for Integrated Communication and Computation , 1992, [1992] Proceedings the 19th Annual International Symposium on Computer Architecture.

[10]  Charles L. Seitz,et al.  Myrinet: A Gigabit-per-Second Local Area Network , 1995, IEEE Micro.

[11]  Leonard Kleinrock,et al.  Virtual Cut-Through: A New Computer Communication Switching Technique , 1979, Comput. Networks.

[12]  Jim Griffioen,et al.  A New Design for Distributed Systems: The Remote Memory Model , 1990, USENIX Summer.

[13]  Henry M. Levy,et al.  Limits to low-latency communication on high-speed networks , 1993, TOCS.

[14]  Anna R. Karlin,et al.  Reducing network latency using subpages in a global memory environment , 1996, ASPLOS VII.

[15]  David A. Patterson,et al.  Computer Architecture: A Quantitative Approach , 1969 .

[16]  Michael Dahlin,et al.  Cooperative caching: using remote client memory to improve file system performance , 1994, OSDI '94.