TierStore: A Distributed Filesystem for Challenged Networks in Developing Regions

TierStore is a distributed filesystem that simplifies the development and deployment of applications in challenged network environments, such as those in developing regions. For effective support of bandwidth-constrained and intermittent connectivity, it uses the Delay Tolerant Networking store-and-forward network overlay and a publish/subscribe-based multicast replication protocol. TierStore provides a standard filesystem interface and a single-object coherence approach to conflict resolution which, when augmented with application-specific handlers, is both sufficient for many useful applications and simple to reason about for programmers. In this paper, we show how these properties enable easy adaptation and robust deployment of applications even in highly intermittent networks and demonstrate the flexibility and bandwidth savings of our prototype with initial evaluation results.

[1]  Eric A. Brewer,et al.  Analysis of WWW traffic in Cambodia and Ghana , 2006, WWW '06.

[2]  Lei Gao,et al.  PRACTI Replication , 2006, NSDI.

[3]  Mostafa Ammar,et al.  Multicasting in delay tolerant networks: semantic models and routing algorithms , 2005, WDTN '05.

[4]  Mahadev Satyanarayanan,et al.  Disconnected Operation in the Coda File System , 1999, Mobidata.

[5]  Mark Halsey,et al.  Searching the World Wide Web in Low-Connectivity Communities , 2002 .

[6]  Lakshminarayanan Subramanian,et al.  Rethinking Wireless in the Developing World , 2006, HotNets.

[7]  Werner Vogels,et al.  Dynamo: amazon's highly available key-value store , 2007, SOSP.

[8]  Don de Savigny,et al.  Fixing health systems : the issue, the research, the lessons , 2004 .

[9]  Eric A. Brewer,et al.  The case for technology in developing regions , 2005, Computer.

[10]  Marvin Theimer,et al.  Flexible update propagation for weakly consistent replication , 1997, SOSP.

[11]  Ben Y. Zhao,et al.  Awarded Best Student Paper! - Pond: The OceanStore Prototype , 2003 .

[12]  Kevin R. Fall,et al.  A delay-tolerant network architecture for challenged internets , 2003, SIGCOMM '03.

[13]  Paul Mackerras,et al.  The rsync algorithm , 1996 .

[14]  Eric A. Brewer,et al.  Harvest, yield, and scalable tolerant systems , 1999, Proceedings of the Seventh Workshop on Hot Topics in Operating Systems.

[15]  Sanjeev Khanna,et al.  Agreeing to Agree: Conflict Resolution for Optimistically Replicated Data , 2006, DISC.

[16]  Dirk Kutscher,et al.  Scalable DTN distribution over uni-directional links , 2007, NSDR '07.

[17]  Matthew Kam,et al.  The Case for Technology in Developing Regions , 2005, Computer.

[18]  Eric A. Brewer,et al.  Dtwiki: a disconnection and intermittency tolerant wiki , 2008, WWW.

[19]  Vinton G. Cerf,et al.  Delay-Tolerant Networking Architecture , 2007, RFC.

[20]  Bowei Du,et al.  TierStore: a distributed file system for challenged network in developing regions , 2008 .

[21]  Marvin Theimer,et al.  Managing update conflicts in Bayou, a weakly connected replicated storage system , 1995, SOSP.

[22]  Peter L. Reiher,et al.  Rumor: Mobile Data Access Through Optimistic Peer-to-Peer Replication , 1998, ER Workshops.

[23]  Dan Walsh,et al.  Design and implementation of the Sun network filesystem , 1985, USENIX Conference Proceedings.

[24]  Ashish Goel,et al.  Perspectives on optimistically replicated, peer‐to‐peer filing , 1998, Softw. Pract. Exp..

[25]  Eyal de Lara,et al.  Haggle: Clean-slate networking for mobile devices , 2007 .

[26]  Alex Pentland,et al.  DakNet: rethinking connectivity in developing nations , 2004, Computer.

[27]  Mike Hibler,et al.  An integrated experimental environment for distributed systems and networks , 2002, OPSR.

[28]  Ben Y. Zhao,et al.  Pond: The OceanStore Prototype , 2003, FAST.

[29]  Benjamin C. Pierce,et al.  What's in Unison? A Formal Specification and Reference Implementation of a File Synchronizer , 2004 .