ViewBox: integrating local file systems with cloud storage services

Cloud-based file synchronization services have become enormously popular in recent years, both for their ability to synchronize files across multiple clients and for the automatic cloud backups they provide. However, despite the excellent reliability that the cloud back-end provides, the loose coupling of these services and the local file system makes synchronized data more vulnerable than users might believe. Local corruption may be propagated to the cloud, polluting all copies on other devices, and a crash or untimely shutdown may lead to inconsistency between a local file and its cloud copy. Even without these failures, these services cannot provide causal consistency. To address these problems, we present ViewBox, an integrated synchronization service and local file system that provides freedom from data corruption and inconsistency. ViewBox detects these problems using ext4-cksum, a modified version of ext4, and recovers from them using a user-level daemon, cloud helper, to fetch correct data from the cloud. To provide a stable basis for recovery, ViewBox employs the view manager on top of ext4-cksum. The view manager creates and exposes views, consistent in-memory snapshots of the file system, which the synchronization client then uploads. Our experiments show that ViewBox detects and recovers from both corruption and inconsistency, while incurring minimal overhead.

[1]  Andrea C. Arpaci-Dusseau,et al.  A File Is Not a File: Understanding the I/O Behavior of Apple Desktop Applications , 2012, TOCS.

[2]  Andrea C. Arpaci-Dusseau,et al.  *-Box: Towards Reliability and Consistency in Dropbox-like File Synchronization Services , 2013, HotStorage.

[3]  Andrea C. Arpaci-Dusseau,et al.  Parity Lost and Parity Regained , 2008, FAST.

[4]  Andrea C. Arpaci-Dusseau,et al.  An analysis of data corruption in the storage stack , 2008, TOS.

[5]  Aiko Pras,et al.  Inside dropbox: understanding personal cloud storage services , 2012, Internet Measurement Conference.

[6]  Stephen C. Tweedie,et al.  Journaling the Linux ext2fs Filesystem , 2008 .

[7]  Wei Hu,et al.  Scalability in the XFS File System , 1996, USENIX Annual Technical Conference.

[8]  James Lau,et al.  File System Design for an NFS File Server Appliance , 1994, USENIX Winter.

[9]  Dawson R. Engler,et al.  Bugs as deviant behavior: a general approach to inferring errors in systems code , 2001, SOSP.

[10]  Josef Bacik,et al.  BTRFS: The Linux B-Tree Filesystem , 2013, TOS.

[11]  Andrea C. Arpaci-Dusseau,et al.  IRON file systems , 2005, SOSP '05.

[12]  Erez Zadok,et al.  Extending File Systems Using Stackable Templates , 1999, USENIX Annual Technical Conference, General Track.

[13]  Andrea C. Arpaci-Dusseau,et al.  Optimistic crash consistency , 2013, SOSP.

[14]  Ben Y. Zhao,et al.  Efficient Batched Synchronization in Dropbox-Like Cloud Storage Services , 2013, Middleware.

[15]  John Wilkes,et al.  Seneca: remote mirroring done write , 2003, USENIX Annual Technical Conference, General Track.

[16]  Andrea C. Arpaci-Dusseau,et al.  Zettabyte reliability with flexible end-to-end data integrity , 2013, 2013 IEEE 29th Symposium on Mass Storage Systems and Technologies (MSST).

[17]  Junfeng Yang,et al.  An empirical study of operating systems errors , 2001, SOSP.

[18]  Andrea C. Arpaci-Dusseau,et al.  A file is not a file: understanding the I/O behavior of Apple desktop applications , 2011, SOSP 2011.

[19]  Andrea C. Arpaci-Dusseau,et al.  ROOT: replaying multithreaded traces with resource-oriented ordering , 2013, SOSP.

[20]  YangJunfeng,et al.  An empirical study of operating systems errors , 2001 .

[21]  Kim-Kwang Raymond Choo,et al.  Google Drive: Forensic analysis of data remnants , 2014, J. Netw. Comput. Appl..

[22]  Brian Beach Simple Storage Service , 2014 .