Random choices for churn resilient load balancing in peer-to-peer networks

Peer-to-peer (P2P) networks based on consistent hashing functions have an inherent load uneven distribution problem. Things are even worse in unstructured P2P systems. The objective of load balancing in P2P networks is to balance the workload of the network nodes in proportion to their capacity so as to eliminate traffic bottleneck. It is challenging because of the dynamic nature of overlay networks and time-varying load characteristics. Random choices schemes can balance load effectively while incurring only a small overhead, making such schemes appealing for practical systems. Existing theoretical work analyzing properties of random choices algorithms can not be applied in the highly dynamic and heterogeneous P2P systems. In this paper, we characterize the behaviors of randomized search schemes in the general P2P environment. We extend the supermarket model by investigating the impact of node heterogeneity and churn to the load distribution in P2P networks. We prove that by using d-way random choices schemes, the length of the longest queue in P2P systems with heterogeneous nodal capacity and node churn for d ges 2 is clog logn/logd + O(1) with high probability, where c is a constant.

[1]  Hermann Wagner,et al.  Auf der Heide , 2022 .

[2]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[3]  Ravi Jain,et al.  An Experimental Study of the Skype Peer-to-Peer VoIP System , 2005, IPTPS.

[4]  David E. Culler,et al.  Hood: a neighborhood abstraction for sensor networks , 2004, MobiSys '04.

[5]  Eli Upfal,et al.  Balanced Allocations , 1999, SIAM J. Comput..

[6]  John Kubiatowicz,et al.  Handling churn in a DHT , 2004 .

[7]  Ben Y. Zhao,et al.  Tapestry: a resilient global-scale overlay for service deployment , 2004, IEEE Journal on Selected Areas in Communications.

[8]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[9]  Richard M. Karp,et al.  Load Balancing in Structured P2P Systems , 2003, IPTPS.

[10]  Seif Haridi,et al.  A Statistical Theory of Chord Under Churn , 2005, IPTPS.

[11]  Gurmeet Singh Manku,et al.  Symphony: Distributed Hashing in a Small World , 2003, USENIX Symposium on Internet Technologies and Systems.

[12]  Michael Mitzenmacher,et al.  The Power of Two Choices in Randomized Load Balancing , 2001, IEEE Trans. Parallel Distributed Syst..

[13]  Michael Mitzenmacher,et al.  On the Analysis of Randomized Load Balancing Schemes , 1997, SPAA '97.

[14]  Richard M. Karp,et al.  Load balancing in dynamic structured P2P systems , 2004, IEEE INFOCOM 2004.

[15]  David R. Karger,et al.  Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web , 1997, STOC '97.

[16]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[17]  Alan Weiss,et al.  Large Deviations For Performance Analysis: Queues, Communication and Computing , 1995 .

[18]  Richard M. Karp,et al.  Load balancing in dynamic structured peer-to-peer systems , 2006, Perform. Evaluation.

[19]  Brighten Godfrey,et al.  Heterogeneity and load balance in distributed hash tables , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[20]  Friedhelm Meyer auf der Heide,et al.  Randomized protocols for low-congestion circuit routing in multistage interconnection networks , 1998, STOC '98.

[21]  Jeffrey Considine,et al.  Geometric generalizations of the power of two choices , 2004, SPAA '04.

[22]  Cheng-Zhong Xu,et al.  Locality-Aware and Churn-Resilient Load-Balancing Algorithms in Structured Peer-to-Peer Networks , 2007, IEEE Transactions on Parallel and Distributed Systems.

[23]  David R. Karger,et al.  Wide-area cooperative storage with CFS , 2001, SOSP.

[24]  Friedhelm Meyer auf der Heide,et al.  Dynamic Load Balancing in Distributed Hash Tables , 2005, IPTPS.

[25]  Daniel Stutzbach,et al.  Understanding churn in peer-to-peer networks , 2006, IMC '06.

[26]  David R. Karger,et al.  Chord: a scalable peer-to-peer lookup protocol for internet applications , 2003, TNET.

[27]  Devavrat Shah,et al.  Load balancing with memory , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[28]  Cheng-Zhong Xu,et al.  Elastic Routing Table with Provable Performance for Congestion Control in DHT Networks , 2006, 26th IEEE International Conference on Distributed Computing Systems (ICDCS'06).

[29]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[30]  Yiming Hu,et al.  Efficient, proximity-aware load balancing for DHT-based P2P systems , 2005, IEEE Transactions on Parallel and Distributed Systems.

[31]  Moni Naor,et al.  Viceroy: a scalable and dynamic emulation of the butterfly , 2002, PODC '02.

[32]  Jeffrey Considine,et al.  Simple Load Balancing for Distributed Hash Tables , 2003, IPTPS.

[33]  David R. Karger,et al.  Simple Efficient Load-Balancing Algorithms for Peer-to-Peer Systems , 2004, SPAA '04.

[34]  Hector Garcia-Molina,et al.  Designing a super-peer network , 2003, Proceedings 19th International Conference on Data Engineering (Cat. No.03CH37405).

[35]  Zheng Zhang,et al.  Building topology-aware overlays using global soft-state , 2003, 23rd International Conference on Distributed Computing Systems, 2003. Proceedings..

[36]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[37]  Shipeng Li,et al.  Distributed Segment Tree: Support of Range Query and Cover Query over DHT , 2006, IPTPS.

[38]  Adam Shwartz,et al.  Large Deviations For Performance Analysis , 2019 .

[39]  Guihai Chen,et al.  Cycloid: a constant-degree and lookup-efficient P2P overlay network , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[40]  Leonard Kleinrock,et al.  Proportional Replication in Peer-to-Peer Networks , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[41]  Guihai Chen,et al.  Cycloid: A Constant-Degree and Lookup-Efficient P2P Overlay Network , 2004, IPDPS.

[42]  Ben Y. Zhao,et al.  An Infrastructure for Fault-tolerant Wide-area Location and Routing , 2001 .

[43]  Gurmeet Singh Manku,et al.  Decentralized algorithms using both local and random probes for P2P load balancing , 2005, SPAA '05.

[44]  Dmitri Loguinov,et al.  Modeling Heterogeneous User Churn and Local Resilience of Unstructured P2P Networks , 2006, Proceedings of the 2006 IEEE International Conference on Network Protocols.

[45]  George Giakkoupis,et al.  A scheme for load balancing in heterogenous distributed hash tables , 2005, PODC '05.

[46]  Ellen W. Zegura,et al.  How to model an internetwork , 1996, Proceedings of IEEE INFOCOM '96. Conference on Computer Communications.

[47]  Francis C. M. Lau,et al.  Load balancing in parallel computers - theory and practice , 1996, The Kluwer international series in engineering and computer science.