A load balanced directory for distributed shared memory objects

We present MultiBend, a novel distributed directory protocol for shared objects, suitable for large-scale distributed shared memory systems that use d -dimensional mesh-based topologies, where d ? 2 . Each shared object has an owner node that can modify its value. The ownership may change by moving the object from one node to another in response to move requests. The value of an object can be read by other nodes with lookup requests. MultiBend balances the load of the network edges and nodes by forwarding each move or lookup request and response along a path consisting of multiple bends in the mesh. Using an oblivious routing protocol, the multi-bend paths have a small number of overlaps which helps to reduce the maximum edge and node utilization to achieve load balancing. At the same time, MultiBend achieves small stretch for the total path length of any sequence of move requests, compared to the total optimal path length. MultiBend guarantees O ( d 2 log n ) approximation for the load, and O ( d log n ) approximation for the stretch due to move requests, where n is the number of nodes in the mesh network. It also guarantees O ( d 2 ) approximation for the stretch of lookup requests. We evaluate MultiBend with simulations using various sequences of move and lookup operations in a 16 i? 16 nodes 2-dimensional mesh network. We compare the simulation results to other protocols which are not tailored for load balancing and we find that our protocol is better by as much as the factor of 6.85 in terms of congestion in the worst-case. To the best of our knowledge, this is the first distributed shared memory directory protocol that considers the network load balancing aspect and achieves good approximation ratio for both the load and the stretch. A novel load balanced directory for distributed shared memory objects is proposed.It is suitable for d -dimensional mesh-based topologies with n nodes and d ? 2 .It attains O ( d 2 log n ) load approximation and O ( d log n ) stretch approximation.Experimental results confirm the load balancing and low stretch benefits.Previous protocols only considered stretch and cannot control network load.

[1]  Gokarna Sharma,et al.  An Analysis Framework for Distributed Hierarchical Directories , 2013, Algorithmica.

[2]  Amith R. Mamidala,et al.  Looking under the hood of the IBM Blue Gene/Q network , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[3]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM 2001.

[4]  Gokarna Sharma,et al.  Distributed transactional memory for general networks , 2012, 2012 IEEE 26th International Parallel and Distributed Processing Symposium.

[5]  Thomas F. La Porta,et al.  Mitigating Performance Degradation in Congested Sensor Networks , 2008, IEEE Transactions on Mobile Computing.

[6]  Mohamed Naimi,et al.  A Log(N) Distributed Mutual Exclusion Algorithm Based on Path Reversal , 1996, J. Parallel Distributed Comput..

[7]  Steven L. Scott,et al.  The Cray T3E Network: Adaptive Routing in a High Performance 3D Torus , 1996 .

[8]  Philip Heidelberger,et al.  Blue Gene/L torus interconnection network , 2005, IBM J. Res. Dev..

[9]  William N. Scherer,et al.  Advanced contention management for dynamic software transactional memory , 2005, PODC '05.

[10]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[11]  Michael Luby.  A Simple Parallel Algorithm for the Maximal Independent Set Problem , 1986, SIAM J. Comput..

[12]  Hagit Attiya,et al.  A Provably Starvation-Free Distributed Directory Protocol , 2010, SSS.

[13]  Alex Zelikovsky,et al.  Improved Steiner tree approximation in graphs , 2000, SODA '00.

[14]  Rachid Guerraoui,et al.  Toward a theory of transactional contention managers , 2005, PODC '05.

[15]  Gokarna Sharma,et al.  Towards Load Balanced Distributed Transactional Memory , 2012, Euro-Par.

[16]  Hagit Attiya,et al.  Transactional Contention Management as a Non-Clairvoyant Scheduling Problem , 2010, Algorithmica.

[17]  Paul Feautrier,et al.  A New Solution to Coherence Problems in Multicache Systems , 1978, IEEE Transactions on Computers.

[18]  Noga Alon,et al.  Lower Bounds on the Competitive Ratio for Mobile User Tracking and Distributed Job Scheduling , 1994, Theor. Comput. Sci..

[19]  Mohammad Taghi Hajiaghayi,et al.  Oblivious network design , 2006, SODA '06.

[20]  Binoy Ravindran,et al.  Brief Announcement: Relay: A Cache-Coherence Protocol for Distributed Transactional Memory , 2009, OPODIS.

[21]  Anant Agarwal,et al.  Directory-based cache coherence in large-scale multiprocessors , 1990, Computer.

[22]  Ben Y. Zhao,et al.  Tapestry: a resilient global-scale overlay for service deployment , 2004, IEEE Journal on Selected Areas in Communications.

[23]  Ye Sun,et al.  Distributed transactional memory for metric-space networks , 2005, Distributed Computing.

[24]  Costas Busch,et al.  Optimal Oblivious Path Selection on the Mesh , 2008, IEEE Trans. Computers.

[25]  Donald Yeung,et al.  THE MIT ALEWIFE MACHINE: A LARGE-SCALE DISTRIBUTED-MEMORY MULTIPROCESSOR , 1991 .

[26]  Robert Morris,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM 2001.

[27]  Bruce M. Maggs,et al.  Exploiting locality for data management in systems of limited bandwidth , 1997, Proceedings 38th Annual Symposium on Foundations of Computer Science.

[28]  Edith Cohen,et al.  Optimal oblivious routing in polynomial time , 2003, STOC '03.

[29]  Maurice Herlihy,et al.  The Arrow Distributed Directory Protocol , 1998, DISC.