Toward Eidetic Distributed File Systems

We propose a new point in the design space of versioning and provenance-aware file systems in which the entire operating system, not just the file system, supports such functionality. We leverage deterministic record-and-replay to substitute computation for data. This leads to a new file system design where the log of nondeterministic inputs, not file data, is the fundamental unit of persistent storage. We outline a distributed storage system design based on these principles and describe the challenges we foresee for achieving our vision.

[1]  Mahadev Satyanarayanan,et al.  Exploiting weak connectivity in a distributed file system , 1996 .

[2]  Lenin Ravindranath,et al.  Nectar: Automatic Management of Data and Computation in Datacenters , 2010, OSDI.

[3]  Kiran-Kumar Muniswamy-Reddy,et al.  Causality-based versioning , 2009, TOS.

[4]  Amin Vahdat,et al.  Transparent Result Caching , 1997, USENIX Annual Technical Conference.

[5]  Herbert Bos,et al.  Paranoid Android: versatile protection for smartphones , 2010, ACSAC '10.

[6]  Raghupathy Sivakumar,et al.  Mimic: raw activity shipping for file synchronization in mobile file systems , 2004, MobiSys '04.

[7]  David Mazières,et al.  A low-bandwidth network file system , 2001, SOSP.

[8]  Erez Zadok,et al.  A Versatile and User-Oriented Versioning File System , 2004, FAST.

[9]  Galen C. Hunt,et al.  Shielding Applications from an Untrusted Cloud with Haven , 2014, OSDI.

[10]  Siddhartha Annapureddy,et al.  Shark: scaling file servers via cooperative caching , 2005, NSDI.

[11]  Margo I. Seltzer,et al.  Provenance-Aware Storage Systems , 2006, USENIX ATC, General Track.

[12]  Andrea C. Arpaci-Dusseau,et al.  Explicit Control in the Batch-Aware Distributed File System , 2004, NSDI.

[13]  Craig A. N. Soules,et al.  Connections: using context to enhance file search , 2005, SOSP '05.

[14]  Ion Stoica,et al.  ODR: output-deterministic replay for multicore debugging , 2009, SOSP '09.

[15]  Michael Chow,et al.  Eidetic Systems , 2014, OSDI.

[16]  James Lau,et al.  File System Design for an NFS File Server Appliance , 1994, USENIX Winter.

[17]  David Mazières,et al.  Replication, history, and grafting in the Ori file system , 2013, SOSP.

[18]  Norman C. Hutchinson,et al.  Deciding when to forget in the Elephant file system , 1999, SOSP.

[19]  Samuel T. King,et al.  ReVirt: enabling intrusion analysis through virtual-machine logging and replay , 2002, OPSR.

[20]  Yuanyuan Zhou,et al.  PRES: probabilistic replay with execution sketching on multiprocessors , 2009, SOSP '09.

[21]  Kwong-Sak Leung,et al.  Operation Shipping for Mobile File Systems , 2002, IEEE Trans. Computers.