EpiChord: parallelizing the chord lookup algorithm with reactive routing state management

EpiChord is a DHT lookup algorithm that demonstrates that we can remove the O(logn)-state-per-node restriction on existing DHT topologies to achieve significantly better lookup performance and resilience using a novel reactive routing state maintenance strategy that amortizes network maintenance costs into existing lookups and by issuing parallel queries. Our technique allows us to design a new class of unlimited-state-per-node DHTs that is able to adapt naturally to a wide range of lookup workloads. EpiChord is able to achieve O(1)-hop lookup performance under lookup-intensive workloads and at least O(logn)-hop lookup performance under churn-intensive workloads even in the worst case (though it is expected to perform better on average). Our simulations show that our approach can reduce both lookup latencies and path lengths by a factor of 3 by issuing only 3 queries asynchronously in parallel per lookup. Furthermore, we show that we are able to achieve this result with minimal additional communication overhead and the number of messages generated per lookup is in general no more than that for the corresponding sequential chord lookup algorithm.

[1]  Anjali Gupta,et al.  Efficient Routing for Peer-to-Peer Overlays , 2004, NSDI.

[2]  Stefan Saroiu,et al.  A Measurement Study of Peer-to-Peer File Sharing Systems , 2001 .

[3]  James H. Burrows,et al.  Secure Hash Standard , 1995 .

[4]  Robert Tappan Morris,et al.  Practical, distributed network coordinates , 2004, Comput. Commun. Rev..

[5]  Mor Harchol-Balter,et al.  Resource discovery in distributed networks , 1999, PODC '99.

[6]  Robert Tappan Morris,et al.  Designing a DHT for Low Latency and High Throughput , 2004, NSDI.

[7]  Ben Y. Zhao,et al.  An Infrastructure for Fault-tolerant Wide-area Location and Routing , 2001 .

[8]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[9]  Sriram Ramabhadran,et al.  Workshop Report for the 3rd International Workshop on Peer-to-Peer Systems (IPTPS 2004) , 2004, IPTPS.

[10]  Krishna P. Gummadi,et al.  The impact of DHT routing geometry on resilience and proximity , 2003, SIGCOMM '03.

[11]  Ben Y. Zhao,et al.  Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and , 2001 .

[12]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[13]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[14]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[15]  Erik D. Demaine,et al.  EpiChord: parallelizing the chord lookup algorithm with reactive routing state management , 2004, ICON.

[16]  David R. Karger,et al.  Koorde: A Simple Degree-Optimal Distributed Hash Table , 2003, IPTPS.

[17]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[18]  Robert Tappan Morris,et al.  Comparing the Performance of Distributed Hash Tables Under Churn , 2004, IPTPS.

[19]  Gurmeet Singh Manku,et al.  Routing networks for distributed hash tables , 2003, PODC '03.

[20]  Gurmeet Singh Manku,et al.  Symphony: Distributed Hashing in a Small World , 2003, USENIX Symposium on Internet Technologies and Systems.

[21]  Hui Zhang,et al.  Towards global network positioning , 2001, IMW '01.

[22]  Feng Zhou.  Understanding Chord Performance and Topology-aware OVerlay Construction for Chord , 2003 .

[23]  Indranil Gupta,et al.  Kelips: Building an Efficient and Stable P2P DHT through Increased Memory and Background Overhead , 2003, IPTPS.

[24]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[25]  Moni Naor,et al.  Viceroy: a scalable and dynamic emulation of the butterfly , 2002, PODC '02.

[26]  David R. Karger,et al.  Chord: a scalable peer-to-peer lookup protocol for internet applications , 2003, TNET.

[27]  Stefan Savage,et al.  Structured superpeers: leveraging heterogeneity to provide constant-time lookup , 2003, Proceedings the Third IEEE Workshop on Internet Applications. WIAPP 2003.

[28]  Michael B. Jones,et al.  SkipNet: A Scalable Overlay Network with Practical Locality Properties , 2003, USENIX Symposium on Internet Technologies and Systems.