The HP AutoRAID hierarchical storage system

Con@uring redundant disk arrays is a black art. To configure an array properly, a system administrator must understand the details of both the array and the workload it will support. Incorrect understanding of either, or changes in the workload over time, can lead to poor performance, We present a solution to this problem: a two-level storage hierarchy implemented inside a single disk-array controller. In the upper level of this hierarchy, two copies of active data are stored to provide full redundancy and excellent performance. In the lower level, RAID 5 parity protection is used to provide excellent storage cost for inactive data, at somewhat lower performance. The technology we describe in this article, known as HP AutoRAID, automatically and transparently manages migration of data blocks between these two levels as access patterns change. The result is a fully redundant storage system that is extremely easy to use, is suitable for a wide variety of workloads, is largely insensitive to dynamic workload changes, and performs much better than disk arrays with comparable numbers of spindles and much larger amounts of front-end RAM cache, Because the implementation of the HP AutoRAID technology is almost entirely in software, the additional hardware cost for these benefits is very small. We describe the HP AutoRAID technology in detail, provide performance data for an embodiment of it in a storage array, and summarize the results of simulation studies used to choose algorithms implemented in the array.

[1]  Anahí Gallardo Velázquez,et al.  Conference , 1969, Journal of Neuroscience Methods.

[2]  Shivakumar Venkataraman,et al.  The TickerTAIP parallel RAID architecture , 1993, ISCA '93.

[3]  J. Menon,et al.  Methods for improved update performance of disk arrays , 1992, Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences.

[4]  Ethan L. Miller,et al.  File migration on the Cray Y-MP at the National Center for Atmospheric Research , 1991 .

[5]  M.S. McDonald,et al.  Improving file system performance by dynamically restructuring disk space , 1989, Eighth Annual International Phoenix Conference on Computers and Communications. 1989 Conference Proceedings.

[6]  Carl Staelin,et al.  Idleness is Not Sloth , 1995, USENIX.

[7]  Frederick W. Clegg,et al.  The hp-ux operating system on hp precision architecture computers , 1966 .

[8]  Garth A. Gibson,et al.  RAID: high-performance, reliable secondary storage , 1994, CSUR.

[9]  Pratap N. Misra.  Capacity Analysis of the Mass Storage System , 1981, IBM Syst. J..

[10]  John K. Ousterhout,et al.  Tcl and the Tk Toolkit , 1994 .

[11]  Fred Douglis,et al.  Beating the I/O bottleneck: a case for log-structured file systems , 1989, OPSR.

[12]  Bruce McNutt.  Background Data Movement in a Log-Structured Disk Subsystem , 1993, IBM J. Res. Dev..

[13]  John Wilkes,et al.  UNIX Disk Access Patterns , 1993, USENIX Winter.

[14]  John Wilkes,et al.  An introduction to disk drive modeling , 1994, Computer.

[15]  Randy H. Katz,et al.  Introduction to redundant arrays of inexpensive disks (RAID) , 1989, Digest of Papers. COMPCON Spring 89. Thirty-Fourth IEEE Computer Society International Conference: Intellectual Leverage.

[16]  R. B. Bunt,et al.  Dynamic file management techniques , 1988, Seventh Annual International Phoenix Conference on Computers an Communications. 1988 Conference Proceedings.

[17]  H KatzRandy,et al.  A case for redundant arrays of inexpensive disks (RAID) , 1988 .

[18]  Carl Staelin,et al.  An Implementation of a Log-Structured File System for UNIX , 1993, USENIX Winter.

[19]  Theresa A. Burkes,et al.  Adaptive Hierarchical RAID: A Solution To The RAID 5 Write Problem , 1995, Int. CMG Conference.

[20]  Randy H. Katz,et al.  A case for redundant arrays of inexpensive disks (RAID) , 1988, SIGMOD '88.

[21]  John T. Kohl,et al.  HighLight: Using a Log-structured File System for Tertiary Storage Management , 1993, USENIX Winter.

[22]  Peter M. Chen,et al.  Striping in a RAID level 5 disk array , 1995, SIGMETRICS '95/PERFORMANCE '95.

[23]  Carl Staelin,et al.  "Tcl cures 98.3% of all known simulation configuration problems , 1994 .

[24]  Alan Jay Smith,et al.  Optimization of I/O systems by cache disks and file migration: A summary , 1981, Perform. Evaluation.

[25]  Mendel Rosenblum,et al.  The design and implementation of a log-structured file system , 1991, SOSP '91.

[26]  Robert G. Reynolds,et al.  Minimizing Mean Seek Distance in Mirrored Disk Systems by Cylinder Remapping , 1994, Perform. Evaluation.

[27]  Rich Friedrich,et al.  The Implications of Distributed Data in a Commercial Environment on the Design of Hierarchical Storage Management , 1994, Perform. Evaluation.

[28]  Carla Schlatter Ellis,et al.  Directory Reference Patterns in Hierarchical File Systems , 1989, IEEE Trans. Knowl. Data Eng..

[29]  Jai Menon,et al.  The architecture of a fault-tolerant cached RAID controller , 1993, ISCA '93.

[30]  Vince Cate.  Two levels of filesystem hierarchy on one disk , 1990 .

[31]  Masaru Kitsuregawa,et al.  Dynamic parity stripe reorganizations for RAID5 disk arrays , 1994, Proceedings of 3rd International Conference on Parallel and Distributed Information Systems.

[32]  Jai Menon,et al.  Comparison of sparing alternatives for disk arrays , 1992, ISCA '92.

[33]  John Wilkes,et al.  Disk scheduling algorithms based on rotational position , 1991 .

[34]  Randy H. Katz,et al.  Robo-line Storage: Low Latency, High Capacity Storage Systems over , 1991 .

[35]  Sara McMains,et al.  File System Logging versus Clustering: A Performance Comparison , 1995, USENIX.

[36]  Gary M. King,et al.  Storage Hierarchies , 1989, IBM Syst. J..

[37]  Wilson C. Hsieh,et al.  The logical disk: a new approach to improving file systems , 1994, SOSP '93.

[38]  Margo I. Seltzer,et al.  Disk Scheduling Revisited , 1990 .

[39]  Sanjeev Setia,et al.  Optimal Write Batch Size in Log-Structured File Systems , 1994, Comput. Syst..

[40]  Butler W. Lampson,et al.  On-line data compression in a log-structured file system , 1992, ASPLOS V.

[41]  Jack P. Gelb.  System-Managed Storage , 1989, IBM Syst. J..

[42]  Kenneth Salem,et al.  Adaptive block rearrangement , 1993, TOCS.

[43]  Mary Baker,et al.  Non-volatile memory for fast, reliable file systems , 1992, ASPLOS V.

[44]  Margo I. Seltzer,et al.  Heuristic Cleaning Algorithms in Log-Structured File Systems , 1995, USENIX.

[45]  Garth A. Gibson,et al.  Parity declustering for continuous operation in redundant disk arrays , 1992, ASPLOS V.

[46]  Mark Taunton,et al.  Compressed Executables: An Exercise in Thinking Small , 1991, USENIX Summer.

[47]  Jim Gray,et al.  A census of Tandem system availability between 1985 and 1990 , 1990 .

[48]  Chris Ruemmler,et al.  Disk Shuffling , 1991 .