Self-* Storage: Brick-based Storage with Automated Administration (CMU-CS-03-178)

This white paper describes a new project exploring the design and implementation of “self-* storage systems:” self-organizing, self-configuring, self-tuning, self-healing, self-managing systems of storage bricks. Borrowing organizational ideas from corporate structure and automation technologies from AI and control systems, we hope to dramatically reduce the administrative burden currently faced by data center administrators. Further, compositions of lower cost components can be utilized, with available resources collectively used to achieve high levels of reliability, availability, and performance.

[1]  Randy H. Katz,et al.  A case for redundant arrays of inexpensive disks (RAID) , 1988, SIGMOD '88.

[2]  Jeffrey D. Case,et al.  Simple network management protocol , 1995 .

[3]  Jack P. Gelb.  System-Managed Storage , 1989, IBM Syst. J..

[4]  Jeffrey D. Case,et al.  Simple Network Management Protocol (SNMP) , 1990, RFC.

[5]  Marshall T. Rose.  Convention for defining traps for use with the SNMP , 1991, RFC.

[6]  Garth A. Gibson,et al.  RAID: high-performance, reliable secondary storage , 1994, CSUR.

[7]  Randy H. Katz,et al.  Patterson: "raid: high-performance, reliable secondary storage , 1994 .

[8]  James Lau,et al.  File System Design for an NFS File Server Appliance , 1994, USENIX Winter.

[9]  Gregory R. Ganger,et al.  Generating Representative Synthetic Workloads: An Unsolved Problem , 1995 .

[10]  Elizabeth Shriver,et al.  Attribute-managed storage , 1995 .

[11]  Carl Staelin,et al.  Idleness is Not Sloth , 1995, USENIX.

[12]  Jim Zelenka,et al.  A cost-effective, high-bandwidth storage architecture , 1998, ASPLOS VIII.

[13]  Andrew V. Goldberg,et al.  A prototype implementation of archival Intermemory , 1999, DL '99.

[14]  Gregory R. Ganger,et al.  Automated Disk Drive Characterization , 1999 .

[15]  Pradeep K. Khosla,et al.  Survivable Information Storage Systems , 2000, Computer.

[16]  Mark Russinovich,et al.  Inside Microsoft Windows 2000 , 2000 .

[17]  Gregory R. Ganger,et al.  Towards higher disk head utilization: extracting free bandwidth from busy disk drives , 2000, OSDI.

[18]  Amin Vahdat,et al.  Interposed request routing for scalable network storage , 2000, TOCS.

[19]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[20]  Petr Jan Horn,et al.  Autonomic Computing: IBM's Perspective on the State of Information Technology , 2001 .

[21]  John Wilkes,et al.  Traveling to Rome: QoS Specifications for Automated Storage System Management , 2001, IWQoS.

[22]  Arif Merchant,et al.  A modular, analytical throughput model for modern disk arrays , 2001, MASCOTS 2001, Proceedings Ninth International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[23]  Arif Merchant,et al.  Minerva: An automated resource provisioning tool for large-scale storage systems , 2001, TOCS.

[24]  E. Anderson.  HPL – SSP – 2001 – 4 : Simple table-based modeling of storage devices , 2001 .

[25]  Joseph Hall,et al.  An Experimental Study of Data Migration Algorithms , 2001, WAE.

[26]  Hai Jin,et al.  A Case for Redundant Arrays of Inexpensive Disks (RAID) , 2002 .

[27]  Michael K. Reiter,et al.  Decentralized Storage Consistency via Versioning Servers , 2002 .

[28]  Gregory R. Ganger,et al.  Freeblock Scheduling Outside of Disk Firmware , 2002, FAST.

[29]  Gregory R. Ganger,et al.  Cuckoo: Layered Clustering for NFS , 2002 .

[30]  Steve R. Kleiman,et al.  SnapMirror: File-System-Based Asynchronous Mirroring for Disaster Recovery , 2002, FAST.

[31]  Christos Faloutsos,et al.  Capturing the spatio-temporal behavior of real traffic data , 2002, Perform. Evaluation.

[32]  Eric Anderson,et al.  Proceedings of the Fast 2002 Conference on File and Storage Technologies Hippodrome: Running Circles around Storage Administration , 2022 .

[33]  Kimberly Keeton,et al.  Automating data dependability , 2002, EW 10.

[34]  Eric Anderson,et al.  Selecting RAID Levels for Disk Arrays , 2002, FAST.

[35]  Chenyang Lu,et al.  Proceedings of the Fast 2002 Conference on File and Storage Technologies Aqueduct: Online Data Migration with Performance Guarantees , 2022 .

[36]  Craig A. N. Soules,et al.  A Two-Tiered Software Architecture for Automated Tuning of Disk Layouts (CMU-CS-03-130) , 2003 .

[37]  Kimberly Keeton,et al.  Automatic design of dependable data storage systems , 2003 .

[38]  John Wilkes,et al.  Data Services - From Data to Containers , 2003, FAST.

[39]  Gregory R. Ganger,et al.  A Human Organization Analogy for Self-* Systems , 2003 .

[40]  Michael K. Reiter,et al.  Efficient Consistency for Erasure-coded Data via Versioning Servers (CMU-CS-03-127) , 2003 .

[41]  Arif Merchant,et al.  FAB: Enterprise Storage Systems on a Shoestring , 2003, HotOS.

[42]  Craig A. N. Soules,et al.  Metadata Efficiency in Versioning File Systems , 2003, FAST.

[43]  Jim Gray,et al.  A Conversation with Jim Gray , 2003, ACM Queue.

[44]  Craig A. N. Soules,et al.  Self-securing storage: protecting data in compromised systems , 2000, Foundations of Intrusion Tolerant Systems, 2003 [Organically Assured and Survivable Information Systems].

[45]  Kanishk Jain.  Object-based Storage , 2022 .