Differentiated storage services

This article presents a Differentiated Storage Services architecture for file and storage systems. By classifying data at the block-level, a filesystem can request that different classes of data (e.g., file, directory, executable, text) be handled with different policies (e.g., low-latency versus highbandwidth), and it is left to the storage system to enforce these policies. Our approach assumes that an I/O classifier can be included in-band with each I/O request (e.g., using a field in the SCSI block command set) and that the policy for each class can be specified out-of-band through the management interface of the storage system. We describe our prototypes based on Linux Ext3, Windows NTFS, and hybrid storage systems composed of rotating and solid-state disks. With very little modification, filesystems can identify latency sensitive I/O classes (e.g., small files, directories, metadata, and the journal) and request that the storage system provision the solid-state storage for just these classes; and this is simply one of many possibilities. As part of our ongoing work, across a variety of file and storage systems, we are exploring other policies and mechanisms that can be used to improve application performance, reliability, and security.

[1]  Jeanna Matthews,et al.  Intel® Turbo Memory: Nonvolatile disk caches in the storage hierarchy of mainstream computer systems , 2008, TOS.

[2]  Gregory R. Ganger,et al.  Object-based storage , 2003, IEEE Commun. Mag..

[3]  Ohad Rodeh,et al.  zFS - a scalable distributed file system using object disks , 2003, 20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings..

[4]  Margo I. Seltzer,et al.  Tracking Back References in a Write-Anywhere File System , 2010, FAST.

[5]  Xiaodong Zhang,et al.  Understanding intrinsic characteristics and system implications of flash memory based solid state drives , 2009, SIGMETRICS '09.

[6]  Samir Khuller,et al.  Algorithms for data migration with cloning , 2003, SIAM J. Comput..

[7]  Ravi Wijayaratne,et al.  Providing QOS guarantees for disk I/O , 2000, Multimedia Systems.

[8]  Prashant J. Shenoy,et al.  A practical learning-based approach for dynamic storage bandwidth allocation , 2003, IWQoS'03.

[9]  Abraham Silberschatz,et al.  Operating System Concepts , 1983 .

[10]  Gala Yadgar,et al.  Karma: Know-It-All Replacement for a Multilevel Cache , 2007, FAST.

[11]  P. Krishnan,et al.  Flash memory file caching for mobile computers , 1994, 1994 Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences.

[12]  Erez Zadok,et al.  Type-safe disks , 2006, OSDI '06.

[13]  David Flynn,et al.  DFS: A file system for virtualized flash storage , 2010, TOS.

[14]  Nimrod Megiddo,et al.  Outperforming LRU with an adaptive replacement cache algorithm , 2004, Computer.

[15]  Michael Stonebraker,et al.  Operating system support for database management , 1981, CACM.

[16]  Gerhard Weikum,et al.  The LRU-K page replacement algorithm for database disk buffering , 1993, SIGMOD Conference.

[17]  Kaladhar Voruganti,et al.  Polus: Growing Storage QoS Management Beyond a "4-Year Old Kid" , 2004, FAST.

[18]  Christos Faloutsos,et al.  Storage device performance prediction with CART models , 2004, The IEEE Computer Society's 12th Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, 2004. (MASCOTS 2004). Proceedings..

[19]  Wolfgang Effelsberg,et al.  Principles of database buffer management , 1984, TODS.

[20]  Xiaoyun Zhu,et al.  Triage: Performance differentiation for storage systems using adaptive control , 2005, TOS.

[21]  Xiaoning Ding,et al.  DiskSeen: Exploiting Disk Layout and Access History to Enhance I/O Prefetch , 2007, USENIX Annual Technical Conference.

[22]  Gregory R. Ganger,et al.  Ursa minor: versatile cluster-based storage , 2005, FAST'05.

[23]  Paul Barham,et al.  A fresh approach to file system quality of service , 1997, Proceedings of 7th International Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV '97).

[24]  Margo I. Seltzer,et al.  File classification in self-* storage systems , 2004 .

[25]  Gregory R. Ganger,et al.  Modeling the relative fitness of storage , 2007, SIGMETRICS '07.

[26]  Abraham Silberschatz,et al.  Operating System Concepts, 5th Edition , 1994 .

[27]  Andrea C. Arpaci-Dusseau,et al.  Association Proceedings of the Third USENIX Conference on File and Storage Technologies San Francisco , CA , USA March 31 – April 2 , 2004 , 2004 .

[28]  Christopher Frost,et al.  Better I/O through byte-addressable, persistent memory , 2009, SOSP '09.

[29]  Chenyang Lu,et al.  Proceedings of the Fast 2002 Conference on File and Storage Technologies Aqueduct: Online Data Migration with Performance Guarantees , 2022 .

[30]  Yuanyuan Zhou,et al.  The Multi-Queue Replacement Algorithm for Second Level Buffer Caches , 2001, USENIX Annual Technical Conference, General Track.

[31]  Abraham Silberschatz,et al.  Operating Systems Concepts , 2005 .

[32]  Antony I. T. Rowstron,et al.  Migrating server storage to SSDs: analysis of tradeoffs , 2009, EuroSys '09.

[33]  Gregory R. Ganger,et al.  Track-Aligned Extents: Matching Access Patterns to Disk Drive Characteristics , 2002, FAST.

[34]  J. T. Robinson,et al.  Data cache management using frequency-based replacement , 1990, SIGMETRICS '90.

[35]  Arif Merchant,et al.  A modular, analytical throughput model for modern disk arrays , 2001, MASCOTS 2001, Proceedings Ninth International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[36]  Sang Lyul Min,et al.  LRFU: A Spectrum of Policies that Subsumes the Least Recently Used and Least Frequently Used Policies , 2001, IEEE Trans. Computers.

[37]  Vagelis Hristidis,et al.  BORG: Block-reORGanization for Self-optimizing Storage Systems , 2009, FAST.

[38]  Kang G. Shin,et al.  FS2: dynamic data replication in free disk space for improving disk performance and energy consumption , 2005, SOSP '05.

[39]  Andrea C. Arpaci-Dusseau,et al.  Semantically-Smart Disk Systems , 2003, FAST.

[40]  Andrea C. Arpaci-Dusseau,et al.  Information and control in gray-box systems , 2001, SOSP.

[41]  John Wilkes,et al.  Traveling to Rome: QoS Specifications for Automated Storage System Management , 2001, IWQoS.

[42]  Arif Merchant,et al.  Proportional-Share Scheduling for Distributed Storage Systems , 2007, FAST.

[43]  Feng Chen,et al.  Hystor: making the best use of solid state drives in high performance storage systems , 2011, ICS '11.

[44]  Dennis Shasha,et al.  2Q: A Low Overhead High Performance Buffer Management Replacement Algorithm , 1994, VLDB.

[45]  Gregory R. Ganger,et al.  Argon: Performance Insulation for Shared Storage Servers , 2007, FAST.

[46]  Andrea C. Arpaci-Dusseau,et al.  Life or Death at Block-Level , 2004, OSDI.

[47]  Andrea C. Arpaci-Dusseau,et al.  Awarded Best Student Paper! -- Improving Storage System Availability with D-GRAID , 2004 .

[48]  Dutch T. Meyer,et al.  A study of practical deduplication , 2011, TOS.

[49]  Song Jiang,et al.  CLOCK-Pro: An Effective Improvement of the CLOCK Replacement , 2005, USENIX Annual Technical Conference, General Track.

[50]  Erez Zadok,et al.  DHIS: discriminating hierarchical storage , 2009, SYSTOR '09.

[51]  Song Jiang,et al.  LIRS: an efficient low inter-reference recency set replacement policy to improve buffer cache performance , 2002, SIGMETRICS '02.

[52]  Geoffrey H. Kuenning,et al.  Conquest: Better Performance Through a Disk/Persistent-RAM Hybrid File System , 2002, USENIX Annual Technical Conference, General Track.

[53]  Adam Leventhal,et al.  Flash storage memory , 2008, CACM.

[54]  Lidong Zhou,et al.  Transactional Flash , 2008, OSDI.