DhtFlex: A Flexible Approach to Enable Efficient Atomic Data Management Tailored for Structured Peer-to-Peer Overlays

Distributed hash tables (DHTs) build on structured peer-to-peer overlays because of their inherent high scalability and resilience against node failures. Combined with additional replication strategies such systems promise high availability for published data resources. However, regarding the support for atomic data operations replication comes at the cost of maintaining consistency. Thus, most DHT like systems either focus on immutable data resources or if supported at all they reinvent the wheel for their storage needs strictly focussing on mutable data and their own narrow application domain. What lacks is a generic but efficient solution to enable flexible consistent data operations for replicated data trimmed for highly concurrent and fluctuating environments. In this paper we introduce DhtFlex, a fault-tolerant distributed algorithm tailored for the needs of a DHT and optimized for the consistent management of replicated data resources in such environments. DhtFlex is supposed to serve as a generic building block to any underlying structured peer-to-peer overlay. It imposes an annotated resource concept to typify replicated data. DhtFlex enables efficient support for immutable as well as for optimized atomic operations on mutable data resources.

[1]  Steffen Rusitschka,et al.  The Resource Management Framework: A System for Managing Metadata in Decentralized Networks Using Peer-to-Peer Technology , 2002, AP2PC.

[2]  Krishna P. Gummadi,et al.  King: estimating latency between arbitrary internet end hosts , 2002, IMW '02.

[3]  Leslie Lamport,et al.  The part-time parliament , 1998, TOCS.

[4]  Keith W. Ross,et al.  A Measurement Study of a Large-Scale P2P IPTV System , 2007, IEEE Transactions on Multimedia.

[5]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[6]  Ben Y. Zhao,et al.  OceanStore: An Extremely Wide-Area Storage System , 2002, ASPLOS 2002.

[7]  Henning Schulzrinne,et al.  An Analysis of the Skype Peer-to-Peer Internet Telephony Protocol , 2004, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[8]  Robert Morris,et al.  Etna: A Fault-tolerant Algorithm for Atomic Mutable DHT Data , 2005 .

[9]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[10]  Mahadev Satyanarayanan,et al.  The ITC distributed file system: principles and design , 1985, SOSP 1985.

[11]  Antony I. T. Rowstron,et al.  Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility , 2001, SOSP.

[12]  Nancy A. Lynch,et al.  Atomic Data Access in Distributed Hash Tables , 2002, IPTPS.

[13]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[14]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[15]  Rachid Guerraoui,et al.  Deconstructing paxos , 2003, SIGA.

[16]  B. Cohen,et al.  Incentives Build Robustness in Bit-Torrent , 2003 .

[17]  Dan Walsh,et al.  Design and implementation of the Sun network filesystem , 1985, USENIX Conference Proceedings.

[18]  David R. Karger,et al.  Wide-area cooperative storage with CFS , 2001, SOSP.

[19]  Ben Y. Zhao,et al.  An Infrastructure for Fault-tolerant Wide-area Location and Routing , 2001 .

[20]  Ben Y. Zhao,et al.  Towards a Common API for Structured Peer-to-Peer Overlays , 2003, IPTPS.

[21]  Nancy A. Lynch,et al.  Impossibility of distributed consensus with one faulty process , 1985, JACM.