High performance user level sockets over Gigabit Ethernet

While a number of user-level protocols have been developed to reduce the gap between the performance capabilities of the physical network and the performance actually available, applications that have already been developed on kernel based protocols such as TCP have largely been ignored. There is a need to make these existing TCP applications take advantage of the modern user-level protocols such as EMP or VIA which feature both low-latency and high bandwidth. We have designed, implemented and evaluated a scheme to support such applications written using the sockets API to run over EMP without any changes to the application itself. Using this scheme, we are able to achieve a latency of 28.5 /spl mu/s for the Datagram sockets and 37 /spl mu/s for Data Streaming sockets compared to a latency of 120 /spl mu/s obtained by TCP for 4-byte messages. This scheme attains a peak bandwidth of around 840 Mbps. Both the latency and the throughput numbers are close to those achievable by EMP. The ftp application shows twice as much benefit on our sockets interface while the Web server application shows up to six times performance enhancement as compared to TCP. To the best of our knowledge, this is the first such design and implementation for Gigabit Ethernet.

[1]  W. Richard Stevens,et al.  Unix network programming , 1990, CCRV.

[2]  Calton Pu,et al.  High Performance Sockets and RPC over Virtual Interface (VI) Architecture , 1999, CANPC.

[3]  Dhabaleswar K. Panda,et al.  Design Alternatives for Virtual Interface Architecture and an Implementation on IBM Netfinity NT Cluster , 2001, J. Parallel Distributed Comput..

[4]  Wu-chun Feng,et al.  The Quadrics network (QsNet): high-performance clustering technology , 2001, HOT 9 Interconnects. Symposium on High Performance Interconnects.

[5]  Jin-Soo Kim,et al.  SOVIA: a user-level sockets layer over virtual interface architecture , 2001, Proceedings 42nd IEEE Symposium on Foundations of Computer Science.

[6]  Thorsten von Eicken,et al.  U-Net: a user-level network interface for parallel and distributed computing , 1995, SOSP.

[7]  Rich Seifert Gigabit Ethernet , 2001, LCN.

[8]  P. Wyckoff,et al.  EMP: Zero-Copy OS-Bypass NIC-Driven Gigabit Ethernet Message Passing , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[9]  Corporate The MPI Forum,et al.  MPI: a message passing interface , 1993, Supercomputing '93.

[10]  Charles L. Seitz,et al.  Myrinet: A Gigabit-per-Second Local Area Network , 1995, IEEE Micro.

[11]  A. Chien,et al.  High Performance Messaging on Workstations: Illinois Fast Messages (FM) for Myrinet , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[12]  Dhabaleswar K. Panda,et al.  Can user-level protocols take advantage of multi-CPU NICs? , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[13]  Forum Mpi MPI: A Message-Passing Interface , 1994 .

[14]  Howard Frazier,et al.  Gigabit Ethernet: From 100 to 1000 Mbps , 1999, IEEE Internet Comput..