D1HT: a distributed one hop hash table

Distributed hash tables (DHTs) have been used in a variety of applications, but most DHTs so far have opted to solve lookups with multiple hops, which sacrifices performance in order to keep little routing information and minimize maintenance traffic. In this paper, we introduce D1HT, a novel single hop DHT that is able to maximize performance with reasonable maintenance traffic overhead even for huge and dynamic peer-to-peer (P2P) systems. We formally define the algorithm we propose to detect and notify any membership change in the system, prove its correctness and performance properties, and present a quarantine-like mechanism to reduce the overhead caused by volatile peers. Our analyses show that D1HT has reasonable maintenance bandwidth requirements even for very large systems, while presenting at least twice less bandwidth overhead than previous single hop DHT

[1]  Abhishek Kumar,et al.  On the fundamental tradeoffs between routing table size and network diameter in peer-to-peer networks , 2004, IEEE J. Sel. Areas Commun..

[2]  Moni Naor,et al.  Viceroy: a scalable and dynamic emulation of the butterfly , 2002, PODC '02.

[3]  Florian Schintke,et al.  A framework for self-optimizing grids using P2P components , 2003, 14th International Workshop on Database and Expert Systems Applications, 2003. Proceedings..

[4]  Rodrigo Rodrigues,et al.  When Multi-Hop Peer-to-Peer Routing Matters , 2003 .

[5]  John Kubiatowicz,et al.  Handling churn in a DHT , 2004 .

[6]  Liuba Shrira,et al.  The design of a robust peer-to-peer system , 2002, EW 10.

[7]  David R. Karger,et al.  Looking up data in P2P systems , 2003, CACM.

[8]  Robert Tappan Morris,et al.  Comparing the Performance of Distributed Hash Tables Under Churn , 2004, IPTPS.

[9]  Robert Tappan Morris,et al.  A performance vs. cost framework for evaluating DHT design tradeoffs under churn , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[10]  Robert Morris,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM 2001.

[11]  Angelos D. Keromytis,et al.  SOS: an architecture for mitigating DDoS attacks , 2004, IEEE Journal on Selected Areas in Communications.

[12]  Pierre Fraigniaud,et al.  The content-addressable network d2b , 2003 .

[13]  Rodrigo Rodrigues,et al.  When Multi-hop Peer-to-Peer Lookup Matters , 2004, IPTPS.

[14]  Nazareno Andrade,et al.  Influences on cooperation in BitTorrent communities , 2005, P2PECON '05.

[15]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[16]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[17]  Jacky C. Chu,et al.  Availability and locality measurements of peer-to-peer file systems , 2002, SPIE ITCom.

[18]  Stefan Savage,et al.  Structured superpeers: leveraging heterogeneity to provide constant-time lookup , 2003, Proceedings the Third IEEE Workshop on Internet Applications. WIAPP 2003.

[19]  Robert Tappan Morris,et al.  Bandwidth-efficient management of DHT routing tables , 2005, NSDI.

[20]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[21]  B. Levine,et al.  Exploring the Use of BitTorrent as the Basis for a Large Trace Repository , 2004 .

[22]  Indranil Gupta,et al.  Kelips: Building an Efficient and Stable P2P DHT through Increased Memory and Background Overhead , 2003, IPTPS.

[23]  Christian Scheideler,et al.  A distributed hash table for computational grids , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[24]  Mark Handley,et al.  Application-Level Multicast Using Content-Addressable Networks , 2001, Networked Group Communication.

[25]  Rajmohan Rajaraman,et al.  Accessing Nearby Copies of Replicated Objects in a Distributed Environment , 1997, SPAA '97.

[26]  Michael B. Jones,et al.  SkipNet: A Scalable Overlay Network with Practical Locality Properties , 2003, USENIX Symposium on Internet Technologies and Systems.

[27]  Antony I. T. Rowstron,et al.  Squirrel: a decentralized peer-to-peer web cache , 2002, PODC '02.

[28]  David R. Karger,et al.  Analysis of the evolution of peer-to-peer systems , 2002, PODC '02.

[29]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[30]  Antony I. T. Rowstron,et al.  Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility , 2001, SOSP.

[31]  David R. Karger,et al.  Wide-area cooperative storage with CFS , 2001, SOSP.

[32]  Witold Litwin,et al.  LH*—a scalable, distributed data structure , 1996, TODS.

[33]  Ben Y. Zhao,et al.  Tapestry: a resilient global-scale overlay for service deployment , 2004, IEEE Journal on Selected Areas in Communications.

[34]  David R. Karger,et al.  Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web , 1997, STOC '97.

[35]  David R. Karger,et al.  Koorde: A Simple Degree-Optimal Distributed Hash Table , 2003, IPTPS.

[36]  Honghui Lu,et al.  Peer-to-peer support for massively multiplayer games , 2004, IEEE INFOCOM 2004.

[37]  Emin Gün Sirer,et al.  Beehive: O(1) Lookup Performance for Power-Law Query Distributions in Peer-to-Peer Overlays , 2004, NSDI.

[38]  Stefan Saroiu,et al.  A Measurement Study of Peer-to-Peer File Sharing Systems , 2001 .

[39]  Scott Shenker,et al.  Internet indirection infrastructure , 2002, SIGCOMM 2002.

[40]  Ben Y. Zhao,et al.  Approximate Object Location and Spam Filtering on Peer-to-Peer Systems , 2003, Middleware.

[41]  Johan A. Pouwelse,et al.  The Bittorrent P2P File-Sharing System: Measurements and Analysis , 2005, IPTPS.

[42]  G. Weikum.  Querying the Internet with PIER , 2005 .

[43]  Miguel Castro,et al.  Scribe: a large-scale and decentralized application-level multicast infrastructure , 2002, IEEE J. Sel. Areas Commun..

[44]  Brian D. Noble,et al.  Proceedings of the 5th Symposium on Operating Systems Design and Implementation Pastiche: Making Backup Cheap and Easy , 2022 .

[45]  David R. Karger,et al.  Chord: a scalable peer-to-peer lookup protocol for internet applications , 2003, TNET.

[46]  Ben Y. Zhao,et al.  Pond: The OceanStore Prototype , 2003, FAST.

[47]  Anjali Gupta,et al.  Efficient Routing for Peer-to-Peer Overlays , 2004, NSDI.