Simple Efficient Load-Balancing Algorithms for Peer-to-Peer Systems

Load balancing is a critical issue for the efficient operation of peer-to-peer networks. We give two new load-balancing protocols whose provable performance guarantees are within a constant factor of optimal. Our protocols refine the consistent hashing data structure that underlies the Chord (and Koorde) P2P network. Both preserve Chord's logarithmic query time and near-optimal data migration cost.Consistent hashing is an instance of the distributed hash table (DHT) paradigm for assigning items to nodes in a peer-to-peer system: items and nodes are mapped to a common address space, and nodes have to store all items residing closeby in the address space.Our first protocol balances the distribution of the key address space to nodes, which yields a load-balanced system when the DHT maps items "randomly" into the address space. To our knowledge, this yields the first P2P scheme simultaneously achieving O(log n) degree, O(log n) look-up cost, and constant-factor load balance (previous schemes settled for any two of the three).Our second protocol aims to directly balance the distribution of items among the nodes. This is useful when the distribution of items in the address space cannot be randomized. We give a simple protocol that balances load by moving nodes to arbitrary locations "where they are needed." As an application, we use the last protocol to give an optimal implementation of a distributed data structure for range searches on ordered data.

[1]  Michael A. Padlipsky.  A Perspective on the ARPANET Reference Model , 1983, INFOCOM.

[2]  Aravind Srinivasan,et al.  Chernoff-Hoeffding bounds for applications with limited independence , 1995, SODA '93.

[3]  Anthony P. Reeves,et al.  Strategies for Dynamic Load Balancing on Highly Parallel Computers , 1993, IEEE Trans. Parallel Distributed Syst..

[4]  Ali R. Hurson,et al.  Scheduling and Load Balancing in Parallel and Distributed Systems , 1995 .

[5]  David R. Karger,et al.  Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web , 1997, STOC '97.

[6]  Jong Kim,et al.  Synchronous load balancing in hypercube multicomputers with faulty nodes , 1997, Proceedings 1997 International Conference on Parallel and Distributed Systems.

[7]  Min-You Wu,et al.  On Runtime Parallel Scheduling for Processor Load Balancing , 1997, IEEE Trans. Parallel Distributed Syst..

[8]  Daniel M. Lewin,et al.  Consistent hashing and random trees : algorithms for caching in distributed networks , 1998 .

[9]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[10]  Hwa-Gyeong Im,et al.  Method for Maximal Utilization of Idle Links for Fast Load Balancing , 2001 .

[11]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[12]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[13]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[14]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[15]  Moni Naor,et al.  Viceroy: a scalable and dynamic emulation of the butterfly , 2002, PODC '02.

[16]  David R. Karger,et al.  Analysis of the evolution of peer-to-peer systems , 2002, PODC '02.

[17]  Sajal K. Das,et al.  Adaptive Load-Balancing Algorithms Using Symmetric Broadcast Networks , 2002, J. Parallel Distributed Comput..

[18]  Scott Shenker,et al.  Complex Queries in Dht-based Peer-to-peer Networks , 2002 .

[19]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[20]  Richard M. Karp,et al.  Load Balancing in Structured P2P Systems , 2003, IPTPS.

[21]  Ion Stoica,et al.  Peer-to-Peer Systems II , 2003, Lecture Notes in Computer Science.

[22]  Eli Upfal,et al.  Stability and efficiency of a random local load balancing protocol , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[23]  Matthias Ruhl,et al.  Efficient algorithms for new computational models , 2003 .

[24]  James Aspnes,et al.  Skip graphs , 2003, SODA '03.

[25]  Moni Naor,et al.  Novel architectures for P2P applications: the continuous-discrete approach , 2003, SPAA '03.

[26]  David R. Karger,et al.  Koorde: A Simple Degree-Optimal Distributed Hash Table , 2003, IPTPS.

[27]  Mayank Bawa,et al.  Distributed Balanced Tables: Not Making a Hash of it all , 2003 .

[28]  Richard M. Karp,et al.  A stochastic process on the hypercube with applications to peer-to-peer networks , 2003, STOC '03.

[29]  Cecilia R. Aragon,et al.  Randomized search trees , 2005, Algorithmica.

[30]  G. Weikum.  Querying the Internet with PIER , 2005 .