Routing tradeoffs in dynamic peer-to-peer networks

Distributed Hash Tables (DHTs) are useful tools for building large scale distributed systems. DHTs provide a hash-table-like interface to applications by routing a key to its responsible node among the current set of participating nodes. DHT deployments are characterized by churn, a continuous process of nodes joining and leaving the network. Lookup latency is important to applications that use DHTs to locate data. In order to achieve low latency lookups, each node needs to consume bandwidth to keep its routing tables up to date under churn. A robust DHT should use bandwidth sparingly and avoid overloading the network when the deployment scenario deviates from design assumptions. Ultimately, DHT designers are interested in obtaining best latency lookups using a bounded amount of bandwidth across a wide range of operating environments. This thesis presents a new DHT protocol, Accordion, that achieves this goal. Accordion bounds its overhead traffic according to a user specified bandwidth budget and chooses a routing table size that minimizes lookup latency, balancing the need for both low lookup hop-count and low timeout probability. Accordion employs a unique design for managing routing tables. Nodes acquire new routing entries opportunistically through application lookup traffic. Large bandwidth budgets lead to big routing table and low lookup hop-count. Nodes evict entries that are likely dead based on past statistics of node lifetimes. Short node lifetimes lead to high eviction rates and a small routing table with low maintenance overhead. The routing table size is determined by the equilibrium of the neighbor acquisition and eviction processes. Accordion's automatic table size adjustment allows it to bound its bandwidth use and achieve latencies similar or better than existing manually tuned protocols across a wide range of operating environments.

[1]  James Robertson,et al.  UsenetDHT: A Low Overhead Usenet Server , 2004, IPTPS.

[2]  John Kubiatowicz,et al.  Handling churn in a DHT , 2004 .

[3]  Antony I. T. Rowstron,et al.  Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility , 2001, SOSP.

[4]  Michael Walfish,et al.  Middleboxes No Longer Considered Harmful , 2004, OSDI.

[5]  Robert Tappan Morris,et al.  Comparing the Performance of Distributed Hash Tables Under Churn , 2004, IPTPS.

[6]  Pierre Sens,et al.  Proceedings of Hotos Ix: the 9th Workshop on Hot Topics in Operating Systems Hotos Ix: the 9th Workshop on Hot Topics in Operating Systems Post: a Secure, Resilient, Cooperative Messaging System * , 2022 .

[7]  David E. Culler,et al.  Operating Systems Support for Planetary-Scale Network Services , 2004, NSDI.

[8]  Gurmeet Singh Manku,et al.  Symphony: Distributed Hashing in a Small World , 2003, USENIX Symposium on Internet Technologies and Systems.

[9]  Ion Stoica,et al.  Exploring Tradeoffs in Failure Detection in Routing Overlays , 2003 .

[10]  Moni Naor,et al.  Novel architectures for P2P applications: the continuous-discrete approach , 2003, SPAA '03.

[11]  David Mazières,et al.  Democratizing Content Publication with Coral , 2004, NSDI.

[12]  Raj Jain,et al.  Analysis of the Increase and Decrease Algorithms for Congestion Avoidance in Computer Networks , 1989, Comput. Networks.

[13]  Bryan Ford,et al.  Unmanaged Internet Protocol , 2004, Comput. Commun. Rev..

[14]  Srinivasan Seshan,et al.  Mercury: supporting scalable multi-attribute range queries , 2004, SIGCOMM '04.

[15]  Emin Gün Sirer,et al.  The design and implementation of a next generation name service for the internet , 2004, SIGCOMM '04.

[16]  Indranil Gupta,et al.  Kelips: Building an Efficient and Stable P2P DHT through Increased Memory and Background Overhead , 2003, IPTPS.

[17]  Michael B. Jones,et al.  SkipNet: A Scalable Overlay Network with Practical Locality Properties , 2003, USENIX Symposium on Internet Technologies and Systems.

[18]  Peter Druschel,et al.  Exploiting network proximity in peer-to-peer overlay networks , 2002 .

[19]  Moni Naor,et al.  Viceroy: a scalable and dynamic emulation of the butterfly , 2002, PODC '02.

[20]  Amin Vahdat,et al.  Scalability in adaptive multi-metric overlays , 2004, 24th International Conference on Distributed Computing Systems, 2004. Proceedings..

[21]  Srinivasan Seshan,et al.  Mercury: supporting scalable multi-attribute range queries , 2004, SIGCOMM 2004.

[22]  Jon M. Kleinberg,et al.  Spatial gossip and resource location protocols , 2001, JACM.

[23]  Brighten Godfrey,et al.  OpenDHT: a public DHT service and its uses , 2005, SIGCOMM '05.

[24]  Ben Y. Zhao,et al.  Towards a Common API for Structured Peer-to-Peer Overlays , 2003, IPTPS.

[25]  Dejan Kostic,et al.  Scalability and accuracy in a large-scale network emulator , 2002, CCRV.

[26]  Antony I. T. Rowstron,et al.  Squirrel: a decentralized peer-to-peer web cache , 2002, PODC '02.

[27]  Hari Balakrishnan,et al.  Resilient overlay networks , 2001, SOSP.

[28]  Erik D. Demaine,et al.  EpiChord: parallelizing the chord lookup algorithm with reactive routing state management , 2004, Proceedings. 2004 12th IEEE International Conference on Networks (ICON 2004) (IEEE Cat. No.04EX955).

[29]  David R. Karger,et al.  Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web , 1997, STOC '97.

[30]  Jun Xu.  On the fundamental tradeoffs between routing table size and network diameter in peer-to-peer networks , 2004, IEEE Journal on Selected Areas in Communications.

[31]  Abhijit Bose,et al.  Delayed Internet routing convergence , 2000, SIGCOMM.

[32]  David R. Karger,et al.  Analysis of the evolution of peer-to-peer systems , 2002, PODC '02.

[33]  Jeremy Stribling,et al.  Examining the Tradeoffs of Structured Overlays in a Dynamic Non-Transitive Network , 2003 .

[34]  Brian Kantor,et al.  Network News Transfer Protocol , 1986, RFC.

[35]  Simon S. Lam,et al.  Failure recovery for structured P2P networks: protocol design and performance evaluation , 2004, SIGMETRICS '04/Performance '04.

[36]  Miguel Castro,et al.  Controlling the Cost of Reliability in Peer-to-Peer Overlays , 2003, IPTPS.

[37]  David R. Karger,et al.  Koorde: A Simple Degree-Optimal Distributed Hash Table , 2003, IPTPS.

[38]  Michael Walfish,et al.  Untangling the Web from DNS , 2004, NSDI.

[39]  Emil Sit,et al.  A DHT-based Backup System , 2003 .

[40]  Rodrigo Rodrigues,et al.  Proceedings of Hotos Ix: the 9th Workshop on Hot Topics in Operating Systems Hotos Ix: the 9th Workshop on Hot Topics in Operating Systems High Availability, Scalable Storage, Dynamic Peer Networks: Pick Two , 2022 .

[41]  Robert Tappan Morris,et al.  A performance vs. cost framework for evaluating DHT design tradeoffs under churn , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[42]  V. Paxson,et al.  Growth trends in wide-area TCP connections , 1994, IEEE Network.

[43]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[44]  David Mazières,et al.  A Toolkit for User-Level File Systems , 2001, USENIX Annual Technical Conference, General Track.

[45]  Sugih Jamin,et al.  Inet-3.0: Internet Topology Generator , 2002 .

[46]  Krishna P. Gummadi,et al.  Improving the Reliability of Internet Paths with One-hop Source Routing , 2004, OSDI.

[47]  Robert Morris,et al.  A distributed hash table , 2006 .

[48]  Robert Morris,et al.  Etna: A Fault-tolerant Algorithm for Atomic Mutable DHT Data , 2005 .

[49]  Rodrigo Rodrigues,et al.  When Multi-Hop Peer-to-Peer Routing Matters , 2003 .

[50]  Stefan Savage,et al.  Total Recall: System Support for Automated Availability Management , 2004, NSDI.

[51]  Anjali Gupta,et al.  Efficient Routing for Peer-to-Peer Overlays , 2004, NSDI.

[52]  Robert Tappan Morris,et al.  Designing a DHT for Low Latency and High Throughput , 2004, NSDI.

[53]  Robert Tappan Morris,et al.  Bandwidth-efficient management of DHT routing tables , 2005, NSDI.

[54]  C. Lee Giles,et al.  Digital Libraries and Autonomous Citation Indexing , 1999, Computer.

[55]  Steven D. Gribble,et al.  Robustness in complex systems , 2001, Proceedings Eighth Workshop on Hot Topics in Operating Systems.

[56]  Scott Shenker,et al.  Epidemic algorithms for replicated database maintenance , 1988, OPSR.

[57]  Krishna P. Gummadi,et al.  The impact of DHT routing geometry on resilience and proximity , 2003, SIGCOMM '03.

[58]  Krishna P. Gummadi,et al.  King: estimating latency between arbitrary internet end hosts , 2002, IMW '02.

[59]  Brian D. Noble,et al.  Proceedings of the 5th Symposium on Operating Systems Design and Implementation Pastiche: Making Backup Cheap and Easy , 2022 .

[60]  Ion Stoica,et al.  Non-Transitive Connectivity and DHTs , 2005, WORLDS.

[61]  John Kubiatowicz,et al.  Opendht: a public dht service , 2005 .

[62]  Krishna P. Gummadi,et al.  A measurement study of Napster and Gnutella as examples of peer-to-peer file sharing systems , 2002, CCRV.

[63]  David R. Karger,et al.  Chord: a scalable peer-to-peer lookup protocol for internet applications , 2003, TNET.

[64]  Ben Y. Zhao,et al.  Tapestry: a resilient global-scale overlay for service deployment , 2004, IEEE Journal on Selected Areas in Communications.

[65]  Robbert van Renesse,et al.  A Gossip-Style Failure Detection Service , 2009 .

[66]  Robert Morris,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM 2001.

[67]  Jon M. Kleinberg,et al.  The small-world phenomenon: an algorithmic perspective , 2000, STOC '00.

[68]  Witold Litwin,et al.  LH*—a scalable, distributed data structure , 1996, TODS.

[69]  David E. Culler,et al.  Scalable, distributed data structures for internet service construction , 2000, OSDI.

[70]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[71]  Scott Shenker,et al.  Making gnutella-like P2P systems scalable , 2003, SIGCOMM '03.

[72]  Stefan Savage,et al.  Understanding Availability , 2003, IPTPS.

[73]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[74]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[75]  Scott Shenker,et al.  Internet indirection infrastructure , 2004, TNET.

[76]  Vern Paxson,et al.  Empirically derived analytic models of wide-area TCP connections , 1994, TNET.

[77]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[78]  Dmitri Loguinov,et al.  Graph-theoretic analysis of structured peer-to-peer systems: routing distances and fault resilience , 2003, IEEE/ACM Transactions on Networking.

[79]  Van Jacobson,et al.  Congestion avoidance and control , 1988, SIGCOMM '88.

[80]  Ben Y. Zhao,et al.  Distributed Object Location in a Dynamic Network , 2002, SPAA '02.

[81]  David R. Karger,et al.  OverCite: A Cooperative Digital Research Library , 2005, IPTPS.

[82]  Roy Friedman,et al.  A framework for protocol composition in Horus , 1995, PODC '95.

[83]  Robert Tappan Morris,et al.  Vivaldi: a decentralized network coordinate system , 2004, SIGCOMM '04.

[84]  Krishna P. Gummadi,et al.  Measurement, modeling, and analysis of a peer-to-peer file-sharing workload , 2003, SOSP '03.

[85]  David R. Karger,et al.  Wide-area cooperative storage with CFS , 2001, SOSP.

[86]  Miguel Castro,et al.  Performance and dependability of structured peer-to-peer overlays , 2004, International Conference on Dependable Systems and Networks, 2004.