Brief announcement: building data structures on untrusted peer-to-peer storage with per-participant logs

L is a technique for building multi-user distributed data structures out of untrusted peer-to-peer distributed hash tables (DHTs). L uses multiple logs, one log per participant, to store changes to the data structure. Each participant finds data by consulting all logs, but performs modifications by appending only to its own log. This decentralized structure allows L to maintain meta-data consistency without locking and to isolate users’ changes from each other, an appropriate arrangement for unreliable users. Applications use L to maintain consistent data structures. L interleaves multiple logs deterministically so that decentralized clients can agree on the order of completed operations, even if those operations were issued concurrently. When the data structure is quiescent, L guarantees that clients agree on the state of the data structure. L optionally provides mutual exclusion for applications that need to ensure atomicity for multistep operations. The Ivy file system, built on top of L, demonstrates that L’s consistency guarantees are useful and can be used and implemented efficiently. Regular submission. The first three authors are students. Please consider paper for brief announcement as well.

[1]  John Heidemann,et al.  Architecture of the Ficus Scalable Replicated File System , 1991 .

[2]  Amin Vahdat,et al.  Design and evaluation of a conit-based continuous consistency model for replicated services , 2002, TOCS.

[3]  RosenblumMendel,et al.  The design and implementation of a log-structured file system , 1991 .

[4]  Margo I. Seltzer,et al.  Unifying File System Protection , 2001, USENIX Annual Technical Conference, General Track.

[5]  David R. Karger,et al.  Wide-area cooperative storage with CFS , 2001, SOSP.

[6]  Robert Tappan Morris,et al.  Ivy: a read/write peer-to-peer file system , 2002, OSDI '02.

[7]  Alley Stoughton,et al.  Detection of Mutual Inconsistency in Distributed Systems , 1983, IEEE Transactions on Software Engineering.

[8]  Radek Vingralek,et al.  How to build a trusted database system on untrusted storage , 2000, OSDI.

[9]  Antony I. T. Rowstron,et al.  Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility , 2001, SOSP.

[10]  Mendel Rosenblum,et al.  The design and implementation of a log-structured file system , 1991, SOSP '91.

[11]  Dennis Shasha,et al.  Building secure file systems out of byzantine storage , 2002, PODC '02.

[12]  Marvin Theimer,et al.  Managing update conflicts in Bayou, a weakly connected replicated storage system , 1995, SOSP.

[13]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[14]  N. Lynch,et al.  Atomic Data Access in Content Addressable Networks A Position Paper , 2002 .

[15]  Ben Y. Zhao,et al.  An Infrastructure for Fault-tolerant Wide-area Location and Routing , 2001 .

[16]  Craig A. N. Soules,et al.  Self-securing storage: protecting data in compromised systems , 2000, Foundations of Intrusion Tolerant Systems, 2003 [Organically Assured and Survivable Information Systems].

[17]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.