Design challenges of virtual networks: fast, general-purpose communication

Virtual networks provide applications with the illusion of having their own dedicated, high-performance networks, although network interfaces posses limited, shared resources. We present the design of a large-scale virtual network system and examine the integration of communication programming interface, system resource management, and network interface operation. Our implementation on a cluster of 100 workstations quantifies the impact of virtualization on small message latencies and throughputs, shows full hardware performance is delivered to dedicated applications and time-shared workloads, and shows robust performance under demanding workloads that overcommit interface resources.

[1]  David E. Culler,et al.  Active Message Applications Programming Interface , 1996 .

[2]  Scott Pakin,et al.  Fast messages: efficient, portable communication for workstation clusters and MPPs , 1997, IEEE Concurrency.

[3]  Seth Copen Goldstein,et al.  Active messages: a mechanism for integrating communication and computation , 1998, ISCA '98.

[4]  Leslie Lamport,et al.  Time, clocks, and the ordering of events in a distributed system , 1978, CACM.

[5]  Richard B. Gillett.  Memory Channel Network for PCI , 1996, IEEE Micro.

[6]  Thorsten von Eicken,et al.  U-Net: a user-level network interface for parallel and distributed computing , 1995, SOSP.

[7]  A. Chien,et al.  High Performance Messaging on Workstations: Illinois Fast Messages (FM) for Myrinet , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[8]  Andrea C. Arpaci-Dusseau,et al.  Scheduling with implicit information in distributed systems , 1998, SIGMETRICS '98/PERFORMANCE '98.

[9]  David E. Culler,et al.  Multi Protocol Active Messages on a Cluster of SMP , 1997, ACM/IEEE SC 1997 Conference (SC'97).

[10]  Berny Goodheart,et al.  The magic garden explained - the internals of UNIX System V, release 4: an open systems design , 1994 .

[11]  Thorsten von Eicken,et al.  Low-Latency Communication on the IBM RISC System/6000 SP , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[12]  Thorsten von Eicken,et al.  Incorporating Memory Management into User-Level Network Interfaces , 1997 .

[13]  Mitsuhisa Sato,et al.  PM: An Operating System Coordinated High Performance Communication Library , 1997, HPCN Europe.

[14]  Robert W. Horst.  TNet: A Reliable System Area Network , 1995, IEEE Micro.

[15]  Milon Mackey,et al.  An implementation of the Hamlyn sender-managed interface architecture , 1996, OSDI '96.

[16]  Kai Li,et al.  Retrospective: virtual memory mapped network interface for the SHRIMP multicomputer , 1994, ISCA '98.

[17]  David E. Culler,et al.  Virtual network transport protocols for Myrinet , 1998, IEEE Micro.

[18]  Charles L. Seitz,et al.  Myrinet: A Gigabit-per-Second Local Area Network , 1995, IEEE Micro.

[19]  Lewis W. Tucker,et al.  CMMD: Active Messages on the CM-5 , 1994, Parallel Comput..

[20]  Larry L. Peterson,et al.  Experiences with a high-speed network adaptor: a software perspective , 1994 .

[21]  Hans Werner Meuer,et al.  Top500 Supercomputer Sites , 1997 .

[22]  Richard P. Martin,et al.  LogP Performance Assessment of Fast Network Interfaces , 1995 .

[23]  David E. Culler,et al.  Active message applications programming interface and communication subsystem organization , 1995 .

[24]  D.E. Culler,et al.  Effects Of Communication Latency, Overhead, And Bandwidth In A Cluster Architecture , 1997, Conference Proceedings. The 24th Annual International Symposium on Computer Architecture.

[25]  Cezary Dubnicki,et al.  Stream Sockets on SHRIMP , 1997, CANPC.

[26]  W. Daniel Hillis,et al.  The Network Architecture of the Connection Machine CM-5 , 1996, J. Parallel Distributed Comput..

[27]  Angelos Bilas,et al.  Fast RPC on the SHRIMP Virtual Memory Mapped Network Interface , 1997, J. Parallel Distributed Comput..

[28]  Richard P. Martin,et al.  HPAM: an active message layer for a network of hp workstations , 1994, Symposium Record Hot Interconnects II.

[29]  Andrew A. ChienJanuary.  Fast Messages ( FM ) : E cient , Portable Communication for Workstation Clusters and Massively-Parallel Processors , 1997 .

[30]  Eric A. Brewer,et al.  Remote queues: exposing message queues for optimization and atomicity , 1995, SPAA '95.

[31]  Seth Copen Goldstein,et al.  Active messages: a mechanism for integrating communication and computation , 1998, ISCA '98.

[32]  Chris J. Scheiman,et al.  Experience with active messages on the Meiko CS-2 , 1995, Proceedings of 9th International Parallel Processing Symposium.

[33]  W. Daniel Hillis,et al.  The network architecture of the Connection Machine CM-5 (extended abstract) , 1992, SPAA '92.

[34]  Richard P. Martin,et al.  Assessing Fast Network Interfaces , 1996, IEEE Micro.

[35]  Noah Treuhaft,et al.  Cluster I/O with River: making the fast case common , 1999, IOPADS '99.