Ivy: a read/write peer-to-peer file system

Ivy is a multi-user read/write peer-to-peer file system. Ivy has no centralized or dedicated components, and it provides useful integrity properties without requiring users to fully trust either the underlying peer-to-peer storage system or the other users of the file system.An Ivy file system consists solely of a set of logs, one log per participant. Ivy stores its logs in the DHash distributed hash table. Each participant finds data by consuiting all logs, but performs modifications by appending only to its own log. This arrangement allows Ivy to maintain meta-data consistency without locking. Ivy users can choose which other logs to trust, an appropriate arrangement in a semi-open peer-to-peer system.Ivy presents applications with a conventional file system interface. When the underlying network is fully connected, Ivy provides NFS-like semantics, such as close-to-open consistency. Ivy detects conflicting modifications made during a partition, and provides relevant version information to application-specific conflict resolvers. Performance measurements on a wide-area network show that Ivy is two to three times slower than NFS.

[1]  A. Retrospective,et al.  The UNIX Time-sharing System , 1977 .

[2]  Leslie Lamport,et al.  Time, clocks, and the ordering of events in a distributed system , 1978, CACM.

[3]  Alley Stoughton,et al.  Detection of Mutual Inconsistency in Distributed Systems , 1983, IEEE Transactions on Software Engineering.

[4]  David A. Goldberg,et al.  Design and Implementation of the Sun Network Filesystem , 1985, USENIX Conference Proceedings.

[5]  Ralph C. Merkle,et al.  A Digital Signature Based on a Conventional Encryption Function , 1987, CRYPTO.

[6]  Mahadev Satyanarayanan,et al.  Scale and performance in a distributed file system , 1987, SOSP '87.

[7]  Michael Williams,et al.  Replication in the harp file system , 1991, SOSP '91.

[8]  Disconnected operation in the Coda file system , 1991, SOSP '91.

[9]  John Heidemann,et al.  Architecture of the Ficus Scalable Replicated File System , 1991 .

[10]  Mendel Rosenblum,et al.  The design and implementation of a log-structured file system , 1991, SOSP '91.

[11]  Mahadev Satyanarayanan,et al.  Log-based directory resolution in the Coda file system , 1993, [1993] Proceedings of the Second International Conference on Parallel and Distributed Information Systems.

[12]  John H. Hartman,et al.  The Zebra striped network file system , 1995, TOCS.

[13]  Brent Callaghan,et al.  NFS Version 3 Protocol Specification , 1995, RFC.

[14]  Marvin Theimer,et al.  Managing update conflicts in Bayou, a weakly connected replicated storage system , 1995, SOSP.

[15]  Chandramohan A. Thekkath,et al.  Frangipani: a scalable distributed file system , 1997, SOSP.

[16]  Marvin Theimer,et al.  Dealing with server corruption in weakly consistent, replicated data systems , 1997, MobiCom '97.

[17]  Brian Berliner,et al.  CVS II: Parallelizing Software Dev elopment , 1998 .

[18]  Miguel Oom Temudo de Castro,et al.  Practical Byzantine fault tolerance , 1999, OSDI '99.

[19]  Norman C. Hutchinson,et al.  Deciding when to forget in the Elephant file system , 1999, SOSP.

[20]  David Mazières,et al.  Separating key management from file system security , 1999, SOSP.

[21]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[22]  Radek Vingralek,et al.  How to build a trusted database system on untrusted storage , 2000, OSDI.

[23]  Ian Clarke,et al.  Freenet: A Distributed Anonymous Information Storage and Retrieval System , 2000, Workshop on Design Issues in Anonymity and Unobservability.

[24]  Marvin Theimer,et al.  Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs , 2000, SIGMETRICS '00.

[25]  Margo I. Seltzer,et al.  Unifying File System Protection , 2001, USENIX Annual Technical Conference, General Track.

[26]  Ben Y. Zhao,et al.  An Infrastructure for Fault-tolerant Wide-area Location and Routing , 2001 .

[27]  David Mazières,et al.  A Toolkit for User-Level File Systems , 2001, USENIX Annual Technical Conference, General Track.

[28]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[29]  David R. Karger,et al.  Wide-area cooperative storage with CFS , 2001, SOSP.

[30]  Antony I. T. Rowstron,et al.  Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility , 2001, SOSP.

[31]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[32]  Ben Y. Zhao,et al.  Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and , 2001 .

[33]  Hai Jin,et al.  The Zebra Striped Network File System , 2002 .

[34]  David Mazières,et al.  Fast and secure distributed read-only file system , 2000, TOCS.

[35]  Dennis Shasha,et al.  Building secure file systems out of byzantine storage , 2002, PODC '02.

[36]  Nancy A. Lynch,et al.  Atomic Data Access in Distributed Hash Tables , 2002, IPTPS.

[37]  Hari Balakrishnan,et al.  Resilient overlay networks , 2001, SOSP.

[38]  N. Lynch,et al.  Atomic Data Access in Content Addressable Networks A Position Paper , 2002 .

[39]  Sean Quinlan,et al.  Venti: A New Approach to Archival Storage , 2002, FAST.

[40]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[41]  Craig A. N. Soules,et al.  Self-securing storage: protecting data in compromised systems , 2000, Foundations of Intrusion Tolerant Systems, 2003 [Organically Assured and Survivable Information Systems].