Novel architectures for P2P applications: the continuous-discrete approach

We propose a new approach for constructing P2P networks based on a dynamic decomposition of a continuous space into cells corresponding to processors. We demonstrate the power of these design rules by suggesting two new architectures, one for DHT (Distributed Hash Table) and the other for dynamic expander networks. The DHT network, which we call Distance Halving allows logarithmic routing and load, while preserving constant degrees. It offers an optimal tradeoff between the degree and the dilation in the sense that degree d guarantees a dilation of O(log d n). Another advantage over previous constructions is its relative simplicity. A major new contribution of this construction is a dynamic caching technique that maintains low load and storage even under the occurrence of hot spots. Our second construction builds a network that is guaranteed to be an expander. The resulting topologies are simple to maintain and implement. Their simplicity makes it easy to modify and add protocols. A small variation yields a DHT which is robust against random faults. Finally we show that, using our approach, it is possible to construct any family of constant degree graphs in a dynamic environment, though with worst parameters. Therefore we expect that more distributed data structures could be designed and implemented in a dynamic environment.

[1]  Zvi Galil,et al.  Explicit Constructions of Linear-Sized Superconcentrators , 1981, J. Comput. Syst. Sci..

[2]  Leslie G. Valiant,et al.  A Scheme for Fast Parallel Communication , 1982, SIAM J. Comput..

[3]  Abhiram G. Ranade,et al.  How to emulate shared memory , 1991, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[4]  Bruce M. Maggs,et al.  Expanders might be practical: fast algorithms for routing around faults on multibutterflies , 1989, 30th Annual Symposium on Foundations of Computer Science.

[5]  Arnold L. Rosenberg,et al.  Work-preserving emulations of fixed-connection networks , 1989, STOC '89.

[6]  F. Leighton,et al.  Introduction to Parallel Algorithms and Architectures: Arrays, Trees, Hypercubes , 1991 .

[7]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[8]  Atsuyuki Okabe,et al.  Spatial Tessellations: Concepts and Applications of Voronoi Diagrams , 1992, Wiley Series in Probability and Mathematical Statistics.

[9]  Bruce M. Maggs,et al.  Approximate load balancing on dynamic and asynchronous networks , 1993, STOC.

[10]  Peter B. Danzig,et al.  A Hierarchical Internet Object Cache , 1996, USENIX ATC.

[11]  Rajmohan Rajaraman,et al.  Fast fault-tolerant concurrent access to shared objects , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[12]  David R. Karger,et al.  Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web , 1997, STOC '97.

[13]  Michael K. Reiter,et al.  Probabilistic quorum systems , 1997, PODC '97.

[14]  Michael Luby,et al.  A digital fountain approach to reliable distribution of bulk data , 1998, SIGCOMM '98.

[15]  Bruce M. Maggs,et al.  Real-Time Emulations of Bounded-Degree Networks , 1998, Inf. Process. Lett..

[16]  Krzysztof Diks,et al.  Optimal Adaptive Broadcasting with a Bounded Fraction of Faulty Nodes , 2000, Algorithmica.

[17]  Atsuyuki Okabe,et al.  Spatial Tessellations: Concepts and Applications of Voronoi Diagrams, Second Edition , 2000, Wiley Series in Probability and Mathematical Statistics.

[18]  Jon M. Kleinberg,et al.  The small-world phenomenon: an algorithmic perspective , 2000, STOC '00.

[19]  Ben Y. Zhao,et al.  An Infrastructure for Fault-tolerant Wide-area Location and Routing , 2001 .

[20]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[21]  Ramesh K. Sitaraman,et al.  The power of two random choices: a survey of tech-niques and results , 2001 .

[22]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[23]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[24]  Ben Y. Zhao,et al.  Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and , 2001 .

[25]  David R. Karger,et al.  Building peer-to-peer systems with chord, a distributed lookup service , 2001, Proceedings Eighth Workshop on Hot Topics in Operating Systems.

[26]  Moni Naor,et al.  Viceroy: a scalable and dynamic emulation of the butterfly , 2002, PODC '02.

[27]  David R. Karger,et al.  Analysis of the evolution of peer-to-peer systems , 2002, PODC '02.

[28]  John Kubiatowicz,et al.  Erasure Coding Vs. Replication: A Quantitative Comparison , 2002, IPTPS.

[29]  Jin-Yi Cai Essentially Every Unimodular Matrix Defines an Expander , 2002, Theory of Computing Systems.

[30]  Amos Fiat,et al.  Censorship resistant peer-to-peer content addressable networks , 2002, SODA '02.

[31]  Scott Shenker,et al.  Routing Algorithms for DHTs: Some Open Questions , 2002, IPTPS.

[32]  Stefan Saroiu,et al.  Dynamically Fault-Tolerant Content Addressable Networks , 2002, IPTPS.

[33]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[34]  Joel Friedman,et al.  A proof of Alon's second eigenvalue conjecture and related problems , 2004, ArXiv.

[35]  Pierre Fraigniaud,et al.  Brief announcement: an overview of the content-addressable network D2B , 2003, PODC '03.

[36]  Moni Naor,et al.  A Simple Fault Tolerant Distributed Hash Table , 2003, IPTPS.

[37]  Ching Law,et al.  Distributed construction of random expander graphs , 2003, INFOCOM 2003.

[38]  John Kubiatowicz,et al.  Asymptotically Efficient Approaches to Fault-Tolerance in Peer-to-Peer Networks , 2003, DISC.

[39]  Yossi Azar,et al.  A generic scheme for building overlay networks in adversarial scenarios , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[40]  Dahlia Malkhi,et al.  Estimating network size from local information , 2003, Information Processing Letters.

[41]  Moni Naor,et al.  Scalable and dynamic quorum systems , 2003, PODC.

[42]  M. Larsen Navigating the Cayley graph of SL(2,Z/pZ) , 2003, math/0301147.

[43]  David R. Karger,et al.  Koorde: A Simple Degree-Optimal Distributed Hash Table , 2003, IPTPS.

[44]  Pierre Fraigniaud,et al.  The content-addressable network d2b , 2003 .

[45]  Moni Naor,et al.  Fault-Tolerant Storage in a Dynamic Environment , 2004, DISC.

[46]  David R. Karger,et al.  Simple Efficient Load-Balancing Algorithms for Peer-to-Peer Systems , 2004, SPAA '04.

[47]  Gurmeet Singh Manku,et al.  Balanced binary trees for ID management and load balance in distributed hash tables , 2004, PODC '04.

[48]  Christos Gkantsidis,et al.  Random walks in peer-to-peer networks , 2004, IEEE INFOCOM 2004.

[49]  Edith Cohen,et al.  Balanced-Replication Algorithms for Distribution Trees , 2004, SIAM J. Comput..

[50]  Gurmeet Singh Manku,et al.  Decentralized algorithms using both local and random probes for P2P load balancing , 2005, SPAA '05.