Virtual network transport protocols for Myrinet

Bringing direct and protected network multiprogramming into mainstream cluster computing requires innovations in three key areas: application programming interfaces, network virtualization systems, and lightweight communication protocols for high-speed interconnects. The AM-II API extends traditional active messages with support for client-server computing and facilitates the construction of parallel clients and distributed servers. Our virtual network segment driver enables a large number of arbitrary sequential and parallel applications to access network interface resources directly in a concurrent but fully protected manner. The NIC-to-NIC communication protocols provide reliable and at-most-once message delivery between communication endpoints. The NIC-to-NIC protocols perform well as the number of endpoints and the number of hosts in the cluster are scaled. The flexibility afforded by the underlying protocols enables a diverse set of timely research efforts. Other Berkeley researchers are actively using this system to investigate implicit techniques for the coscheduling of communicating processes, an essential part of high-performance communications in multiprogrammed clusters of uni- and multiprocessor servers. Other researchers are extending the active message protocols described here for clusters of symmetric multiprocessors, using so-called multiprotocol techniques and multiple network interfaces per machine.

[1]  David E. Culler,et al.  Active Message Applications Programming Interface , 1996 .

[2]  Richard P. Martin,et al.  LogP Performance Assessment of Fast Network Interfaces , 1995 .

[3]  David E. Culler,et al.  Active message applications programming interface and communication subsystem organization , 1995 .

[4]  Murad S. Taqqu,et al.  On the Self-Similar Nature of Ethernet Traffic , 1993, SIGCOMM.

[5]  Walter Willinger,et al.  On the self-similar nature of Ethernet traffic , 1993, SIGCOMM '93.

[6]  A. Chien,et al.  High Performance Messaging on Workstations: Illinois Fast Messages (FM) for Myrinet , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[7]  Thorsten von Eicken,et al.  U-Net: a user-level network interface for parallel and distributed computing , 1995, SOSP.

[8]  Milon Mackey,et al.  An implementation of the Hamlyn sender-managed interface architecture , 1996, OSDI '96.

[9]  William R. Crowther,et al.  The interface message processor for the ARPA computer network , 1899, AFIPS '70 (Spring).

[10]  Charles L. Seitz,et al.  Myrinet: A Gigabit-per-Second Local Area Network , 1995, IEEE Micro.

[11]  Kai Li,et al.  Virtual-Memory-Mapped Network Interfaces , 1995, IEEE Micro.

[12]  Bernard Tourancheau,et al.  Protocol design for high performance networking: a Myrinet experience , 1997 .

[13]  Scott Pakin,et al.  Fast messages: efficient, portable communication for workstation clusters and MPPs , 1997, IEEE Concurrency.

[14]  David A. Patterson,et al.  A case for networks of workstations (now) , 1994, Symposium Record Hot Interconnects II.

[15]  Hiroshi Tezuka.  PM : A High-Performance Communication Library for Multi-user Parallel Environments , 1996 .

[16]  David E. Culler,et al.  Multi Protocol Active Messages on a Cluster of SMP , 1997, ACM/IEEE SC 1997 Conference (SC'97).

[17]  Seth Copen Goldstein,et al.  Active messages: a mechanism for integrating communication and computation , 1998, ISCA '98.

[18]  Richard P. Martin,et al.  HPAM: an active message layer for a network of hp workstations , 1994, Symposium Record Hot Interconnects II.

[19]  Andrew A. ChienJanuary.  Fast Messages ( FM ) : E cient , Portable Communication for Workstation Clusters and Massively-Parallel Processors , 1997 .

[20]  Larry L. Peterson,et al.  Experiences with a high-speed network adaptor: a software perspective , 1994 .

[21]  Richard P. Martin,et al.  Assessing Fast Network Interfaces , 1996, IEEE Micro.

[22]  Daniel Shawcross Wilkerson,et al.  System area network mapping , 1997, SPAA '97.

[23]  Andrea C. Arpaci-Dusseau,et al.  Effective distributed scheduling of parallel workloads , 1996, SIGMETRICS '96.