SoftDC: software-based dynamically connected transport

RDMA is increasingly becoming popular not only in HPC but also in data centers where high throughput and low latency are critical requirements. RDMA supports several types of transports, each of which has different characteristics, so that users can choose the right one to meet their requirements. Reliable connected (RC) transport has advantages on usability but disadvantages on scalability while unreliable datagram (UD) transport is scalable but hard to use. Dynamically connected (DC) transport has been newly introduced to address these limitations when using one of the existing transports while delivering both usability and scalability. However, despite all of these merits, DC transport is not yet generally adopted in the related fields due to hardware dependency. To eliminate the hardware dependency, in this paper, we design and implement SoftDC, a totally software-based DC transport. SoftDC uses the basic RDMA primitives to emulate the behavior of DC transport; UD transport for connection and RC transport for data transfer. We build and evaluate a SoftDC transport-based application to prove its effectiveness compared with RC transport-based one. Our experimental results show that our scheme has potential to provide the advantages of both RC and UD transports although our prototype has large connection establishment costs that can be amortized across a large data transfer.

[1]  Haibo Chen,et al.  Fast In-Memory Transaction Processing Using RDMA and HTM , 2017, ACM Trans. Comput. Syst..

[2]  Kang Chen,et al.  RFP: When RPC is Faster than Server-Bypass with RDMA , 2017, EuroSys.

[3]  Dhabaleswar K. Panda,et al.  Designing MPI Library with Dynamic Connected Transport (DCT) of InfiniBand: Early Experiences , 2014, ISC.

[4]  Dhabaleswar K. Panda,et al.  MVAPICH-Aptus: Scalable high-performance multi-transport MPI over InfiniBand , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[5]  Thomas Haynes Network File System (NFS) Version 4 Minor Version 2 Protocol , 2016, RFC.

[6]  David G. Andersen,et al.  Design Guidelines for High Performance RDMA Systems , 2016, USENIX ATC.

[7]  David G. Andersen,et al.  Using RDMA efficiently for key-value services , 2015, SIGCOMM 2015.

[8]  Heon Y. Yeom,et al.  Design and Implementation of Software-based Dynamically Connected Transport , 2018 .

[9]  David G. Andersen,et al.  FaSST: Fast, Scalable and Simple Distributed Transactions with Two-Sided (RDMA) Datagram RPCs , 2016, OSDI.

[10]  Heon Young Yeom,et al.  Design and Implementation of Software-Based Dynamically Connected Transport , 2018, 2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W).

[11]  Dhabaleswar K. Panda,et al.  Scalable Memcached Design for InfiniBand Clusters Using Hybrid Transports , 2012, 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012).

[12]  Miguel Castro,et al.  FaRM: Fast Remote Memory , 2014, NSDI.