A hybrid filesystem for hard disk drives in tandem with flash memory

The traditional hard disk drive (HDD) is often a bottleneck in the overall performance of modern computer systems. With the development of solid state drives (SSD) based on flash memory, new possibilities are available to improve secondary storage performance. In this work, we propose a new hybrid SSD–HDD storage system and a selection of algorithms designed to assign pages across an HDD and an SSD to optimise I/O performance. The hybrid system combines the advantages of the SSD’s fast random seek speed with the sequential access speed and large storage capacity of the HDD to produce significantly improved performance in a variety of situations. We further improve performance by allowing concurrent access across the two types of storage devices. We show the drive assignment problem is NP-complete and accordingly propose effective heuristic solutions. Extensive experiments using both synthetic and real data sets show our system with a small SSD can outperform a striped dual HDD and remain competitive with a dual SSD.

[1]  Stratis Viglas,et al.  Flashing up the storage layer , 2008, Proc. VLDB Endow..

[2]  Sivan Toledo,et al.  Algorithms and data structures for flash memories , 2005, CSUR.

[3]  A. L. Narasimha Reddy,et al.  Umbrella file system: Storage management across heterogeneous devices , 2009, TOS.

[4]  Seung-Ho Lim,et al.  Hashing Directory Scheme for NAND Flash File System , 2007, The 9th International Conference on Advanced Communication Technology.

[5]  Youngjae Kim,et al.  DFTL: a flash translation layer employing demand-based selective caching of page-level address mappings , 2009, ASPLOS.

[6]  Harvey F. Silverman,et al.  Placement of Records on a Secondary Storage Device to Minimize Access Time , 1973, JACM.

[7]  Sang Lyul Min,et al.  A space-efficient flash translation layer for CompactFlash systems , 2002, IEEE Trans. Consumer Electron..

[8]  Jeanna Matthews,et al.  Intel® Turbo Memory: Nonvolatile disk caches in the storage hierarchy of mainstream computer systems , 2008, TOS.

[9]  Arif Merchant,et al.  Awarded Best Paper! - Using MEMS-Based Storage in Disk Arrays , 2003 .

[10]  Young-Jin Kim,et al.  LAST: locality-aware sector translation for NAND flash memory-based storage systems , 2008, OPSR.

[11]  Bo Hong,et al.  Using MEMS-based storage in computer systems---MEMS storage architectures , 2006, TOS.

[12]  C. Kirsch.  Combo Drive : Optimizing Cost and Performance in a Heterogeneous Storage Device , 2009 .

[13]  Kanad Ghose,et al.  yFS: A Journaling File System Design for Handling Large Data Sets with Reduced Seeking , 2003, FAST.

[14]  Mendel Rosenblum,et al.  The design and implementation of a log-structured file system , 1991, SOSP '91.

[15]  Mary Baker,et al.  Non-volatile memory for fast, reliable file systems , 1992, ASPLOS V.

[16]  Sukhan Lee,et al.  MEMS for IT applications , 2001, MHS2001. Proceedings of 2001 International Symposium on Micromechatronics and Human Science (Cat. No.01TH8583).

[17]  Adam Leventhal,et al.  Flash storage memory , 2008, CACM.

[18]  Arif Merchant,et al.  Using MEMS-Based Storage in Disk Arrays , 2003, FAST.

[19]  Mahesh Balakrishnan,et al.  Extending SSD Lifetimes with Disk-Based Write Caches , 2010, FAST.

[20]  Philippe Bonnet,et al.  uFLIP: Understanding Flash IO Patterns , 2009, CIDR.

[21]  Steven Swanson,et al.  Gordon: using flash memory to build fast, power-efficient clusters for data-intensive applications , 2009, ASPLOS.

[22]  Jacob R. Lorch,et al.  A five-year study of file-system metadata , 2007, TOS.

[23]  K. J. Gabriel.  Microelectromechanical systems (MEMS) , 1997, 1997 IEEE Aerospace Conference.

[24]  Kyu Ho Park,et al.  An efficient NAND flash file system for flash memory storage , 2006, IEEE Transactions on Computers.

[25]  Heeseung Jo,et al.  A superblock-based flash translation layer for NAND flash memory , 2006, EMSOFT '06.

[26]  Kyu Ho Park,et al.  JFTL: A flash translation layer based on a journal remapping for flash memory , 2009, TOS.

[27]  Eitan Bachmat,et al.  Dynamic data reallocation in disk arrays , 2007, TOS.

[28]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[29]  Tara M. Madhyastha,et al.  Proceedings of Fast '03: 2nd Usenix Conference on File and Storage Technologies 2nd Usenix Conference on File and Storage Technologies Optimizing Probe-based Storage , 2022 .

[30]  Tei-Wei Kuo,et al.  Efficient management for large-scale flash-memory storage systems with resource conservation , 2005, TOS.

[31]  Tei-Wei Kuo,et al.  Endurance Enhancement of Flash-Memory Storage, Systems: An Efficient Static Wear Leveling Design , 2007, 2007 44th ACM/IEEE Design Automation Conference.

[32]  Divyakant Agrawal,et al.  Towards optimal I/O scheduling for MEMS-based storage , 2003, 20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings..

[33]  Jongmoo Choi,et al.  Block recycling schemes and their cost-based optimization in nand flash memory based storage system , 2007, EMSOFT '07.

[34]  Yanping Zhao,et al.  HyLog: A High Performance Approach to Managing Disk Layout , 2004, FAST.

[35]  Yan Liu,et al.  PMSH: a new algorithm for RAID data migration based on stripe unit heat , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[36]  Geoffrey H. Kuenning,et al.  The Conquest file system: Better performance through a disk/persistent-RAM hybrid design , 2006, TOS.

[37]  Scott A. Brandt,et al.  HeRMES: high-performance reliable MRAM-enabled storage , 2001, Proceedings Eighth Workshop on Hot Topics in Operating Systems.

[38]  Sang-Won Lee,et al.  A log buffer-based flash translation layer using fully-associative sector translation , 2007, TECS.