Serverless network file systems

We propose a new paradigm for network file system design: serverless network file systems. While traditional network file systems rely on a central server machine, a serverless system utilizes workstations cooperating as peers to provide all file system services. Any machine in the system can store, cache, or control any block of data. Our approach uses this location independence, in combination with fast local area networks, to provide better performance and scalability than traditional file systems. Furthermore, because any machine in the system can assume the responsibilities of a failed component, our serverless design also provides high availability via redundatn data storage. To demonstrate our approach, we have implemented a prototype serverless network file system called xFS. Preliminary performance measurements suggest that our architecture achieves its goal of scalability. For instance, in a 32-node xFS system with 32 active clients, each client receives nearly as much read or write throughput as it would see if it were the only active client.

[1]  Alan Jay Smith,et al.  Two Methods for the Efficient Analysis of Memory Address Trace Data , 1977, IEEE Transactions on Software Engineering.

[2]  Bruce Walker,et al.  The LOCUS distributed operating system , 1983, SOSP '83.

[3]  R. S. Fabry,et al.  A fast file system for UNIX , 1984, TOCS.

[4]  David A. Goldberg,et al.  Design and Implementation of the Sun Network Filesystem , 1985, USENIX Conference Proceedings.

[5]  Randy H. Katz,et al.  A case for redundant arrays of inexpensive disks (RAID) , 1988, SIGMOD '88.

[6]  Scale and performance in a distributed file system , 1988, TOCS.

[7]  Michael N. Nelson,et al.  Caching in the Sprite network file system , 1988, TOCS.

[8]  M. L. Scott,et al.  Beyond striping: the bridge multiprocessor file system , 1989, CARN.

[9]  Joel L. Wolf,et al.  The placement optimization program: a practical solution to the disk file assignment problem , 1989, SIGMETRICS '89.

[10]  Michael L. Kazar.  Ubik: replicated servers made easy , 1989, Proceedings of the Second Workshop on Workstation Operating Systems.

[11]  Anoop Gupta,et al.  The directory-based cache coherence protocol for the DASH multiprocessor , 1990, ISCA '90.

[12]  Philip S. Yu,et al.  Policies for efficient memory utilization in a remote caching architecture , 1991, [1991] Proceedings of the First International Conference on Parallel and Distributed Information Systems.

[13]  Michael Williams,et al.  Replication in the harp file system , 1991, SOSP '91.

[14]  Kenneth P. Birman,et al.  Using process groups to implement failure detection in asynchronous environments , 1991, PODC '91.

[15]  Michael Burrows,et al.  Autonet: A High-Speed, Self-Configuring Local Area Network Using Point-to-Point Links , 1991, IEEE J. Sel. Areas Commun..

[16]  Anant Agarwal,et al.  LimitLESS directories: A scalable cache coherence scheme , 1991, ASPLOS IV.

[17]  Mary Baker,et al.  Measurements of a distributed file system , 1991, SOSP '91.

[18]  Fred Douglis,et al.  Transparent process migration: Design alternatives and the sprite implementation , 1991, Softw. Pract. Exp..

[19]  Darrell D. E. Long,et al.  Swift: a storage architecture for large objects , 1991, [1991] Digest of Papers Eleventh IEEE Symposium on Mass Storage Systems.

[20]  D. Culler,et al.  Active Messages: A Mechanism for Integrated Communication and Computation , 1992, [1992] Proceedings the 19th Annual International Symposium on Computer Architecture.

[21]  Non-Volatile Memory for Fast, Reliable File Systems , 1992, ASPLOS.

[22]  Mahadev Satyanarayanan,et al.  Disconnected operation in the Coda File System , 1992, TOCS.

[23]  Mary Baker,et al.  Non-volatile memory for fast, reliable file systems , 1992, ASPLOS V.

[24]  Mendel Rosenblum,et al.  The design and implementation of a log-structured file system , 1991, SOSP '91.

[25]  Erik Hagersten,et al.  DDM - A Cache-Only Memory Architecture , 1992, Computer.

[26]  Seth Copen Goldstein,et al.  Active Messages: A Mechanism for Integrated Communication and Computation , 1992, [1992] Proceedings the 19th Annual International Symposium on Computer Architecture.

[27]  Matthew Addison Blaze.  Caching in large-scale distributed file systems , 1993 .

[28]  Evgenia Smirni,et al.  The KSR1: experimentation and modeling of poststore , 1993, SIGMETRICS '93.

[29]  Dror G. Feitelson,et al.  Overview of the Vesta parallel file system , 1993, CARN.

[30]  Thomas E. Anderson,et al.  xFS: a wide area mass storage file system , 1993, Proceedings of IEEE 4th Workshop on Workstation Operating Systems. WWOS-III.

[31]  Anant Agarwal,et al.  Anatomy of a message in the Alewife multiprocessor , 1993 .

[32]  Shivakumar Venkataraman,et al.  The TickerTAIP parallel RAID architecture , 1993, ISCA '93.

[33]  Carl Staelin,et al.  An Implementation of a Log-Structured File System for UNIX , 1993, USENIX Winter.

[34]  Anant Agarwal,et al.  Anatomy of a message in the Alewife multiprocessor , 1993, ICS '93.

[35]  P. Messina,et al.  Architectural requirements of parallel scientific applications with explicit communication , 1993, ISCA '93.

[36]  Garth A. Gibson,et al.  RAID: high-performance, reliable secondary storage , 1994, CSUR.

[37]  Anoop Gupta,et al.  The Stanford FLASH Multiprocessor , 1994, ISCA.

[38]  Anoop Gupta,et al.  The Stanford FLASH multiprocessor , 1994, ISCA '94.

[39]  Mary Baker.  Fast crash recovery in distributed file systems , 1994 .

[40]  Michael Dahlin,et al.  Cooperative caching: using remote client memory to improve file system performance , 1994, OSDI '94.

[41]  Mahadev Satyanarayanan,et al.  Disconnected Operation in the Coda File System , 1999, Mobidata.

[42]  Jehoshua Bruck,et al.  EVENODD: an optimal scheme for tolerating double disk failures in RAID architectures , 1994, ISCA '94.

[43]  Michael Dahlin,et al.  A quantitative analysis of cache policies for scalable network file systems , 1994, SIGMETRICS.

[44]  Greg Minshall,et al.  An Overview of the NetWare Operating System , 1994, USENIX Winter.

[45]  David E. Culler,et al.  A case for NOW (networks of workstation) , 1995, PODC '95.

[46]  Sara McMains,et al.  File System Logging versus Clustering: A Performance Comparison , 1995, USENIX.

[47]  Charles L. Seitz,et al.  Myrinet: A Gigabit-per-Second Local Area Network , 1995, IEEE Micro.

[48]  Edward K. Lee.  Highly-available, scalable network storage , 1995, Digest of Papers. COMPCON'95. Technologies for the Information Superhighway.

[49]  John H. Hartman,et al.  The Zebra striped network file system , 1995, TOCS.

[50]  Carl Staelin,et al.  The HP AutoRAID hierarchical storage system , 1995, SOSP.

[51]  Margo I. Seltzer,et al.  Heuristic Cleaning Algorithms in Log-Structured File Systems , 1995, USENIX.

[52]  David E. Culler,et al.  A case for NOW (networks of workstation) , 1995, PODC '95.

[53]  Thorsten von Eicken,et al.  U-Net: a user-level network interface for parallel and distributed computing , 1995, SOSP.

[54]  Garret Swart,et al.  The Echo Distributed File System , 1996 .