Load balancing and locality in range-queriable data structures

We describe a load-balancing mechanism for assigning elements to servers in a distributed data structure that supports range queries. The mechanism ensures both load-balancing with respect to an arbitrary load measure specified by the user and geographical locality, assigning elements with similar keys to the same server. Though our mechanism is specifically designed to improve the performance of skip graphs, it can be adapted to provide deterministic, locality-preserving load-balancing to any distributed data structure that orders machines in a ring or line.

[1]  Ola Petersson,et al.  Approximate Indexed Lists , 1998, J. Algorithms.

[2]  Theodore Johnson,et al.  A distributed data-balanced dictionary based on the B-link tree , 1992, Proceedings Sixth International Parallel Processing Symposium.

[3]  Paul F. Dietz,et al.  A Tight Lower Bound for On-line Monotonic List Labeling , 1994, SWAT.

[4]  Helmut Prodinger,et al.  The path length of random skip lists , 1994, Acta Informatica.

[5]  David R. Karger,et al.  Simple Efficient Load Balancing Algorithms for Peer-to-Peer Systems , 2004, IPTPS.

[6]  Csaba D. Tóth,et al.  Uncoordinated Load Balancing and Congestion Games in P2P Systems , 2004, IPTPS.

[7]  J. Ian Munro,et al.  Analysis of the Expected Search Cost in Skip Lists , 1990, SWAT.

[8]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[9]  Christian Scheideler,et al.  Peer-to-peer systems for prefix search , 2003, PODC '03.

[10]  Bobby Bhattacharjee,et al.  Are Virtualized Overlay Networks Too Much of a Good Thing? , 2002, IPTPS.

[11]  Paul F. Dietz,et al.  Lower Bounds for Monotonic List Labeling , 1990, SWAT.

[12]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[13]  Robert D. Blumofe,et al.  Scheduling multithreaded computations by work stealing , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[14]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[15]  Scott Shenker,et al.  Range Queries over DHTs , 2003 .

[16]  Ben Y. Zhao,et al.  Tapestry: a resilient global-scale overlay for service deployment , 2004, IEEE Journal on Selected Areas in Communications.

[17]  Michael B. Jones,et al.  SkipNet: A Scalable Overlay Network with Practical Locality Properties , 2003, USENIX Symposium on Internet Technologies and Systems.

[18]  Scott Shenker,et al.  Complex Queries in Dht-based Peer-to-peer Networks , 2002 .

[19]  Dan E. Willard,et al.  A Density Control Algorithm for Doing Insertions and Deletions in a Sequentially Ordered File in Good Worst-Case Time , 1992, Inf. Comput..

[20]  David R. Karger,et al.  Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web , 1997, STOC '97.

[21]  James Aspnes,et al.  Skip graphs , 2003, SODA '03.

[22]  Alon Itai,et al.  A Sparse Table Implementation of Priority Queues , 1981, ICALP.

[23]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[24]  Richard Cole,et al.  Two Simplified Algorithms for Maintaining Order in a List , 2002, ESA.

[25]  William Pugh,et al.  Skip Lists: A Probabilistic Alternative to Balanced Trees , 1989, WADS.

[26]  David E. Culler,et al.  A blueprint for introducing disruptive technology into the Internet , 2003, CCRV.