iWarp protocol kernel space software implementation

Zero-copy, RDMA, and protocol offload are three very important characteristics of high performance interconnects. Previous networks that made use of these techniques were built upon proprietary, and often expensive, hardware. With the introduction of iWarp, it is now possible to achieve all three over existing low-cost TCP/IP networks. iWarp is a step in the right direction, but requires an expensive RNIC to enable zero-copy, RDMA, and protocol offload. While the hardware is expensive at present, given that iWarp is based on a commodity interconnect, prices fall. In the meantime only the most critical of servers make use of iWarp, but in order to take advantage of the RNIC both sides must be so equipped. It is for this reason that we have implemented the iWarp protocol in software. This allows a server equipped with an RNIC to exploit its advantages even if the client does not have an RNIC. While throughput and latency do not improve by doing this, the server with the RNIC does experience a dramatic reduction in system load. This means that the server is much more scalable, and can handle many more clients than would otherwise be possible with the usual sockets/TCP/IP protocol stack

[1]  Craig Partridge,et al.  When the CRC and TCP checksum disagree , 2000, SIGCOMM.

[2]  Jeff Hilland RDMA Protocol Verbs Specification , 2003 .

[3]  Hyun-Wook Jin,et al.  Supporting iWARP Compatibility and Features for Regular Network Adapters , 2005, 2005 IEEE International Conference on Cluster Computing.

[4]  P. Wyckoff,et al.  EMP: Zero-Copy OS-Bypass NIC-Driven Gigabit Ethernet Message Passing , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[5]  Message Passing Interface Forum MPI: A message - passing interface standard , 1994 .

[6]  Pete Wyckoff,et al.  A Performance Analysis of the Ammasso RDMA Enabled Ethernet Adapter and its iWARP API , 2005, 2005 IEEE International Conference on Cluster Computing.

[7]  Milon Mackey,et al.  An implementation of the Hamlyn sender-managed interface architecture , 1996, OSDI '96.

[8]  Pete Wyckoff,et al.  Design and Implementation of the iWarp Protocol in Software , 2005, IASTED PDCS.

[9]  Hemal Shah,et al.  Direct Data Placement over Reliable Transports , 2007, RFC.

[10]  Renato Recio,et al.  An RDMA Protocol Specification , 2002 .

[11]  Thorsten von Eicken,et al.  U-Net: a user-level network interface for parallel and distributed computing , 1995, SOSP.

[12]  Thomas Stricker,et al.  Speculative defragmentation - a technique to improve the communication software efficiency for Gigabit Ethernet , 2000, Proceedings the Ninth International Symposium on High-Performance Distributed Computing.

[13]  Jim Pinkerton,et al.  DDP/RDMAP Security , 2003 .

[14]  Renato Recio,et al.  Marker PDU Aligned Framing for TCP Specification , 2007, RFC.