Efficient lookup on unstructured topologies

We present LMS, a protocol for efficient lookup on unstructured networks. Our protocol uses a virtual namespace without imposing specific topologies. It is more efficient than existing lookup protocols for unstructured networks, and thus is an attractive alternative for applications in which the topology cannot be structured as a Distributed Hash Table (DHT). We present analytic bounds for the worst-case performance of LMS. Through detailed simulations (with up to 100,000 nodes), we show that the actual performance on realistic topologies is significantly better. We also show in both simulations and a complete implementation (which includes over five hundred nodes) that our protocol is inherently robust against multiple node failures and can adapt its replication strategy to optimize searches according to a specific heuristic. Moreover, the simulation demonstrates the resilience of LMS to high node turnover rates, and that it can easily adapt to orders of magnitude changes in network size. The overhead incurred by LMS is small, and its performance approaches that of DHTs on networks of similar size

[1]  Joel Friedman,et al.  A proof of Alon's second eigenvalue conjecture and related problems , 2004, ArXiv.

[2]  Venugopalan Ramasubramanian,et al.  Beehive: Exploiting Power Law Query Distributions for O(1) Lookup Performance in Peer to Peer Overlays , 2003 .

[3]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[4]  Moni Naor,et al.  Viceroy: a scalable and dynamic emulation of the butterfly , 2002, PODC '02.

[5]  Scott Shenker,et al.  Making gnutella-like P2P systems scalable , 2003, SIGCOMM '03.

[6]  Jacky C. Chu,et al.  Availability and locality measurements of peer-to-peer file systems , 2002, SPIE ITCom.

[7]  Aravind Srinivasan,et al.  Efficient lookup on unstructured topologies , 2007, IEEE J. Sel. Areas Commun..

[8]  Rajmohan Rajaraman,et al.  Compact routing with name independence , 2003, SPAA '03.

[9]  Mikkel Thorup,et al.  Compact name-independent routing with minimum stretch , 2004, SPAA '04.

[10]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[11]  Krishna P. Gummadi,et al.  Measuring and analyzing the characteristics of Napster and Gnutella hosts , 2003, Multimedia Systems.

[12]  Michael K. Reiter,et al.  Crowds: anonymity for Web transactions , 1998, TSEC.

[13]  Ian Clarke,et al.  Protecting Free Expression Online with Freenet , 2002, IEEE Internet Comput..

[14]  Ian T. Foster,et al.  Mapping the Gnutella Network: Macroscopic Properties of Large-Scale Peer-to-Peer Systems , 2002, IPTPS.

[15]  Ben Y. Zhao,et al.  An Infrastructure for Fault-tolerant Wide-area Location and Routing , 2001 .

[16]  Ittai Abraham,et al.  Name independent routing for growth bounded networks , 2005, SPAA '05.

[17]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[18]  Burton H. Bloom,et al.  Space/time trade-offs in hash coding with allowable errors , 1970, CACM.

[19]  Miguel Castro,et al.  Secure routing for structured peer-to-peer overlay networks , 2002, OSDI '02.

[20]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[21]  Michael Mitzenmacher,et al.  Compressed bloom filters , 2001, PODC '01.

[22]  Alan M. Frieze.  Edge-disjoint paths in expander graphs , 2000, SODA '00.

[23]  Song Jiang,et al.  LightFlood: an efficient flooding scheme for file search in unstructured peer-to-peer systems , 2003, 2003 International Conference on Parallel Processing, 2003. Proceedings..

[24]  Albert-László Barabási,et al.  Error and attack tolerance of complex networks , 2000, Nature.

[25]  Rajeev Motwani,et al.  Randomized algorithms , 1996, CSUR.

[26]  Hector Garcia-Molina,et al.  Improving search in peer-to-peer networks , 2002, Proceedings 22nd International Conference on Distributed Computing Systems.

[27]  Sugih Jamin,et al.  Inet-3.0: Internet Topology Generator , 2002 .

[28]  Ittai Abraham,et al.  Probabilistic Quorums for Dynamic Systems , 2003, DISC.

[29]  Aravind Srinivasan,et al.  Randomized Distributed Edge Coloring via an Extension of the Chernoff-Hoeffding Bounds , 1997, SIAM J. Comput..

[30]  Christos Gkantsidis,et al.  Random walks in peer-to-peer networks , 2004, IEEE INFOCOM 2004.

[31]  Hector Garcia-Molina,et al.  YAPPERS: a peer-to-peer lookup service over arbitrary topology , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[32]  John Kubiatowicz,et al.  Probabilistic location and routing , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[33]  Emin Gün Sirer,et al.  Beehive: O(1) Lookup Performance for Power-Law Query Distributions in Peer-to-Peer Overlays , 2004, NSDI.

[34]  Devdatt P. Dubhashi,et al.  Balls and bins: A study in negative dependence , 1996, Random Struct. Algorithms.

[35]  Ibrahim Matta,et al.  On the origin of power laws in Internet topologies , 2000, CCRV.

[36]  Nabil Kahale.  Large Deviation Bounds for Markov Chains , 1997, Comb. Probab. Comput..

[37]  Richard P. Martin,et al.  PlanetP: using gossiping to build content addressable peer-to-peer information sharing communities , 2003, High Performance Distributed Computing, 2003. Proceedings. 12th IEEE International Symposium on.

[38]  Edith Cohen,et al.  Search and replication in unstructured peer-to-peer networks , 2002, ICS '02.

[39]  Ben Y. Zhao,et al.  Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and , 2001 .