A high performance redundancy scheme for cluster file systems

A known issue in the design of striped file systems is their vulnerability to disk failures. In this paper, we address the challenges of augmenting an existing striped file system with traditional RAID redundancy, and propose a novel redundancy scheme designed to maximise disk throughput seen by applications. We implement our new scheme in CSAR, a proof-of-concept redundant file system based on the parallel virtual file system, along with two other well-known schemes. Our tests using both microbenchmarks and representative scientific applications show that our scheme consistently performs as well as the best of the other two schemes. The application-dependent, potentially larger storage occupation of our scheme is justified by current technological trends that put I/O bandwidth at a premium over disk space.

[1]  John H. Hartman,et al.  The Swarm scalable storage system , 1999, Proceedings. 19th IEEE International Conference on Distributed Computing Systems (Cat. No.99CB37003).

[2]  Mendel Rosenblum,et al.  The design and implementation of a log-structured file system , 1991, SOSP '91.

[3]  Michael Stonebraker,et al.  Distributed RAID-a new multiple copy algorithm , 1990, [1990] Proceedings. Sixth International Conference on Data Engineering.

[4]  Hai Jin,et al.  RAID-x: a new distributed disk array for I/O-centric cluster computing , 2000, Proceedings the Ninth International Symposium on High-Performance Distributed Computing.

[5]  Mario Lauria,et al.  A high performance redundancy scheme for cluster file systems , 2003, 2003 Proceedings IEEE International Conference on Cluster Computing.

[6]  Chandramohan A. Thekkath,et al.  Petal: distributed virtual disks , 1996, ASPLOS VII.

[7]  David A. Patterson,et al.  Tertiary Disk: Large Scale Distributed Storage , 1998 .

[8]  Mario Lauria,et al.  CSAR: cluster storage with adaptive redundancy , 2003, 2003 International Conference on Parallel Processing, 2003. Proceedings..

[9]  Eran Gabber,et al.  Data logging: a method for efficient data updates in constantly active RAIDs , 1998, Proceedings 14th International Conference on Data Engineering.

[10]  Rajeev Thakur,et al.  I/O in Parallel Applications: the Weakest Link , 1998, Int. J. High Perform. Comput. Appl..

[11]  Carl Staelin,et al.  The HP AutoRAID hierarchical storage system , 1995, SOSP.

[12]  Garth A. Gibson,et al.  Parity logging disk arrays , 1994, TOCS.

[13]  Darrell D. E. Long,et al.  Swift/RAID: A Distributed RAID System , 1994, Comput. Syst..

[14]  Garth A. Gibson,et al.  RAID: high-performance, reliable secondary storage , 1994, CSUR.

[15]  H. Apte,et al.  Serverless Network File Systems , 2006 .

[16]  John H. Hartman,et al.  The Zebra striped network file system , 1995, TOCS.

[17]  Evgenia Smirni,et al.  Workload Characterization of Input/Output Intensive Parallel Applications , 1997, Computer Performance Evaluation.