EXIO : Enabling globus on rdma networks-a case study with InfiniBand

For its low-latency, high bandwidth, and low CPU utilization, RDMA (Remote Direct Memory Access) has established itself as an effective data movement technology in many networking environments. This includes recent incarnations of InfiniBand-based RDMA on longdistance networks such as UltraScience Net. However, grid runt-time systems, such as the Globus toolkit, are not yet capable of utilizing RDMA for fast data movement. In this paper, we examine the architecture of Globus for the feasibility of enabling RDMA. An I/O architecture EXIO is designed to extend the Globus XIO system and match the characteristics of RDMA in terms of both connection establishment and communication progress. An initial proof-of-concept implementation of EXIO is realized on InfiniBand. Our experimental results demonstrate that, compared to IPoIB and 10GigE, RDMA can significantly improve the performance of Globus, reducing the latency by 32% and increasing the bandwidth by more than three times. In achieving such performance improvements, RDMA dramatically cuts down CPU utilization of Globus clients and servers. These results adequately demonstrate that EXIO is effectively designed and implemented to exploit the benefits of RDMA for Globus.