A low-bandwidth network file system

Users rarely consider running network file systems over slow or wide-area networks, as the performance would be unacceptable and the bandwidth consumption too high. Nonetheless, efficient remote file access would often be desirable over such networks---particularly when high latency makes remote login sessions unresponsive. Rather than run interactive programs such as editors remotely, users could run the programs locally and manipulate remote files through the file system. To do so, however, would require a network file system that consumes less bandwidth than most current file systems.This paper presents LBFS, a network file system designed for low-bandwidth networks. LBFS exploits similarities between files or versions of the same file to save bandwidth. It avoids sending data over the network when the same data can already be found in the server's file system or the client's cache. Using this technique in conjunction with conventional compression and caching, LBFS consumes over an order of magnitude less bandwidth than traditional network file systems on common workloads.

[1]  Brent Callaghan,et al.  NFS Version 3 Protocol Specification , 1995, RFC.

[2]  Hector Garcia-Molina,et al.  Copy detection mechanisms for digital documents , 1995, SIGMOD '95.

[3]  Assar Westerlund,et al.  Arla: a free AFS client , 1998 .

[4]  F TichyWalter The string-to-string correction problem with block moves , 1984 .

[5]  David Wetherall,et al.  A protocol-independent technique for eliminating redundant network traffic , 2000, SIGCOMM.

[6]  Udi Manber,et al.  Finding Similar Files in a Large File System , 1994, USENIX Winter.

[7]  Garret Swart,et al.  A coherent distributed file cache with directory write-behind , 1994, TOCS.

[8]  David Mazières,et al.  Separating key management from file system security , 1999, SOSP.

[9]  Amin Vahdat,et al.  Design and evaluation of a conit-based continuous consistency model for replicated services , 2002, TOCS.

[10]  Ben Y. Zhao,et al.  OceanStore: An Extremely Wide-Area Storage System , 2002, ASPLOS 2002.

[11]  Kwong-Sak Leung,et al.  Operation-based Update Propagation in a Mobile File System , 1999, USENIX Annual Technical Conference, General Track.

[12]  Andrei Z. Broder,et al.  On the resemblance and containment of documents , 1997, Proceedings. Compression and Complexity of SEQUENCES 1997 (Cat. No.97TB100171).

[13]  Walter F. Tichy,et al.  The string-to-string correction problem with block moves , 1984, TOCS.

[14]  Eddie Kohler,et al.  The Click modular router , 1999, SOSP.

[15]  Mahadev Satyanarayanan,et al.  Disconnected Operation in the Coda File System , 1999, Mobidata.

[16]  David R. Cheriton,et al.  Leases: an efficient fault-tolerant mechanism for distributed file cache consistency , 1989, SOSP '89.

[17]  Marvin Theimer,et al.  Flexible update propagation for weakly consistent replication , 1997, SOSP.

[18]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[19]  Assar Westerlund,et al.  The design of a multicast-based distributed file system , 1999, OSDI '99.

[20]  Amin Vahdat,et al.  Design and evaluation of a continuous consistency model for replicated services , 2000, OSDI.

[21]  David Robinson,et al.  NFS version 4 Protocol , 2000, RFC.

[22]  Mahadev Satyanarayanan,et al.  Scale and performance in a distributed file system , 1987, SOSP '87.

[23]  David Mazières,et al.  A Toolkit for User-Level File Systems , 2001, USENIX Annual Technical Conference, General Track.

[24]  Anja Feldmann,et al.  Potential benefits of delta encoding and data compression for HTTP , 1997, SIGCOMM '97.

[25]  Andrew Tridgell,et al.  Efficient Algorithms for Sorting and Synchronization , 1999 .