A Modeling Framework for Reliability of Erasure Codes in SSD Arrays

Emergence of Solid-State Drives (SSDs) have evolved the data storage industry where they are rapidly replacing Hard Disk Drives (HDDs) due to their superiority in performance and power. Meanwhile, SSDs have reliability issues due to bit errors, bad blocks, and bad chips. To help reliability, Redundant Array of Independent Disks (RAID) configurations, originally proposed to increase both performance and reliability of HDDs, are also applied to SSD arrays. However, the conventional reliability models of HDD RAID cannot be intactly applied to SSD arrays, as the nature of failures in SSDs are totally different from HDDs. Previous studies on the reliability of SSD arrays are based on the deprecated SSD failure data, and only focus on limited failure types, device failures, and page failures caused by the bit errors, while recent field studies have reported other failure types including bad blocks and bad chips, and a high correlation between failures. In this paper, we investigate the reliability of SSD arrays using field storage traces and real-system implementation of conventional and emerging erasure codes. The reliability is evaluated by statistical fault injection experiments that post-process the usage logs obtained from the real-system implementation, while the fault/failure attributes are obtained from the state-of-the-art field data by previous works. As a case study, we examine conventional RAID5 and RAID6 and emerging Partial-MDS (PMDS) codes, Sector-Disk (SD) codes, and STAIR codes in terms of both reliability and performance using an open-source software RAID controller, MD (in Linux kernel version 3.10.0-327), and arrays of Samsung 850 Pro SSDs. Our detailed analysis on the data loss breakdown shows that a) emerging erasure codes fail to replace RAID6 in terms of reliability, b) row-wise erasure codes are the most efficient choices for contemporary SSD devices, and c) previous models overestimate the SSD array reliability by up to six orders of magnitude, as they just focus on the coincidence of bad pages (bit errors) and bad chips within a data stripe that holds the minority of root cause of data loss in SSD arrays. Our experiments show that the combination of bad chips with bad blocks is recognized as the major source of data loss in RAID5 and emerging codes (contributing more than 54 and 90 percent of data loss in RAID5 and emerging codes, respectively), while RAID6 remains robust under these failure combinations. Finally, the fault injection results reveal that SSD array reliability, as well as the failure breakdown is significantly correlated with SSD type.

[1]  Chaitali Chakrabarti,et al.  Flexible product code-based ECC schemes for MLC NAND Flash memories , 2011, 2011 IEEE Workshop on Signal Processing Systems (SiPS).

[2]  Hossein Asadi,et al.  Operating system level data tiering using online workload characterization , 2015, The Journal of Supercomputing.

[3]  A. L. Narasimha Reddy,et al.  Does RAID Improve Lifetime of SSD Arrays? , 2016, ACM Trans. Storage.

[4]  Hong Jiang,et al.  Performance impact and interplay of SSD parallelism through advanced commands, allocation strategy and data granularity , 2011, ICS '11.

[5]  Hossein Asadi,et al.  Investigating power outage effects on reliability of solid-state drives , 2018, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[6]  Zili Shao,et al.  Demand-based block-level address mapping in large-scale NAND flash storage systems , 2010, 2010 IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS).

[7]  Jörg Henkel Emerging Memory Technologies , 2017, IEEE Des. Test.

[8]  Cong Xu,et al.  NVSim: A Circuit-Level Performance, Energy, and Area Model for Emerging Nonvolatile Memory , 2012, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[9]  Renhai Chen,et al.  A Block-Level Log-Block Management Scheme for MLC NAND Flash Memory Storage Systems , 2017, IEEE Transactions on Computers.

[10]  S. Wicker Error Control Systems for Digital Communication and Storage , 1994 .

[11]  Gregory R. Ganger,et al.  The DiskSim Simulation Environment Version 4.0 Reference Manual (CMU-PDL-08-101) , 1998 .

[12]  Hossein Pedram,et al.  Using Silent Writes in Low-Power Traffic-Aware ECC , 2011, PATMOS.

[13]  Randy H. Katz,et al.  A case for redundant arrays of inexpensive disks (RAID) , 1988, SIGMOD '88.

[14]  Mario Blaum,et al.  Sector-Disk (SD) Erasure Codes for Mixed Failure Modes in RAID Systems , 2014, TOS.

[15]  Zili Shao,et al.  A Two-Level Caching Mechanism for Demand-Based Page-Level Address Mapping in NAND Flash Memory Storage Systems , 2011, 2011 17th IEEE Real-Time and Embedded Technology and Applications Symposium.

[16]  E. L. Miller,et al.  Building Flexible , Fault-Tolerant Flash-based Storage Systems , 2009 .

[17]  Huawei Li,et al.  ProPRAM: Exploiting the transparent logic resources in Non-Volatile Memory for Near Data Computing , 2015, 2015 52nd ACM/EDAC/IEEE Design Automation Conference (DAC).

[18]  Himabindu Pucha,et al.  Cost Effective Storage using Extent Based Dynamic Tiering , 2011, FAST.

[19]  Zili Shao,et al.  MNFTL: An efficient flash translation layer for MLC NAND flash memory storage systems , 2011, 2011 48th ACM/EDAC/IEEE Design Automation Conference (DAC).

[20]  Hossein Asadi,et al.  Evaluating impact of human errors on the availability of data storage systems , 2017, Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017.

[21]  Paul H. Siegel,et al.  Characterizing flash memory: Anomalies, observations, and applications , 2009, 2009 42nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[22]  Onur Mutlu,et al.  Error patterns in MLC NAND flash memory: Measurement, characterization, and analysis , 2012, 2012 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[23]  Ahmed Amer,et al.  When MTTDLs Are Not Good Enough : Providing Better Estimates of Disk Array Reliability , 2008 .

[24]  Jie Liu,et al.  SSD Failures in Datacenters: What? When? and Why? , 2016, SYSTOR.

[25]  Ronald F. DeMara,et al.  Non-Volatile Memory Trends: Toward Improving Density and Energy Profiles across the System Stack , 2018, Computer.

[26]  Qiang Wu,et al.  A Large-Scale Study of Flash Memory Failures in the Field , 2015, SIGMETRICS 2015.

[27]  Onur Mutlu,et al.  An Analytical Model for Performance and Lifetime Estimation of Hybrid DRAM-NVM Main Memories , 2019, IEEE Transactions on Computers.

[28]  E. D. S. E. Silva,et al.  Transient Solutions for Markov Chains , 2000 .

[29]  Xiaoxia Wu,et al.  Hybrid cache architecture with disparate memory technologies , 2009, ISCA '09.

[30]  Darrell D. E. Long,et al.  A flexible simulation tool for estimating data loss risks in storage arrays , 2013, 2013 IEEE 29th Symposium on Mass Storage Systems and Technologies (MSST).

[31]  Ren-Shuo Liu,et al.  DuraCache: A durable SSD cache using MLC NAND flash , 2013, 2013 50th ACM/EDAC/IEEE Design Automation Conference (DAC).

[32]  Mingqiang Li,et al.  STAIR Codes: A General Family of Erasure Codes for Tolerating Device and Sector Failures , 2014, TOS.

[33]  Onur Mutlu,et al.  ECI-Cache , 2018, Proc. ACM Meas. Anal. Comput. Syst..

[34]  Darrell D. E. Long,et al.  Three-Dimensional Redundancy Codes for Archival Storage , 2013, 2013 IEEE 21st International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems.

[35]  Hossein Asadi,et al.  Dependability Analysis of Data Storage Systems in Presence of Soft Errors , 2019, IEEE Transactions on Reliability.

[36]  Rina Panigrahy,et al.  Design Tradeoffs for SSD Performance , 2008, USENIX ATC.

[37]  Erez Zadok,et al.  Filebench: A Flexible Framework for File System Benchmarking , 2016, login Usenix Mag..

[38]  Zili Shao,et al.  PCM-FTL: A Write-Activity-Aware NAND Flash Memory Management Scheme for PCM-Based Embedded Systems , 2011, 2011 IEEE 32nd Real-Time Systems Symposium.

[39]  Jiri Schindler,et al.  Beyond MTTDL: A Closed-Form RAID 6 Reliability Equation , 2014, TOS.

[40]  Spencer W. Ng,et al.  Disk scrubbing in large archival storage systems , 2004, The IEEE Computer Society's 12th Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, 2004. (MASCOTS 2004). Proceedings..

[41]  Darrell D. E. Long,et al.  Disaster recovery codes: increasing reliability with large-stripe erasure correcting codes , 2007, StorageSS '07.

[42]  Ahmed Amer,et al.  RESAR: Reliable Storage at Exabyte Scale , 2016, 2016 IEEE 24th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS).

[43]  Mario Blaum,et al.  Partial-MDS Codes and Their Application to RAID Type of Architectures , 2012, IEEE Transactions on Information Theory.

[44]  Reza Salkhordeh,et al.  An Efficient Hybrid I/O Caching Architecture Using Heterogeneous SSDs , 2019, IEEE Transactions on Parallel and Distributed Systems.

[45]  D. Long,et al.  Reliability Modelling of Disk Subsystems with Probabilistic Model Checking , 2009 .

[46]  Cheng Li,et al.  Nitro: A Capacity-Optimized SSD Cache for Primary Storage , 2014, USENIX Annual Technical Conference.

[47]  Hossein Pedram,et al.  HVD: horizontal-vertical-diagonal error detecting and correcting code to protect against with soft errors , 2011, Des. Autom. Embed. Syst..

[48]  Hamid Sarbazi-Azad,et al.  A Hybrid Non-Volatile Cache Design for Solid-State Drives Using Comprehensive I/O Characterization , 2016, IEEE Transactions on Computers.

[49]  Jinchun Kim,et al.  Speculative paging for future NVM storage , 2017, MEMSYS.

[50]  Tao Zhang,et al.  RC-NVM: Enabling Symmetric Row and Column Memory Accesses for In-memory Databases , 2018, 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[51]  Chaitali Chakrabarti,et al.  Improving the Reliability of MLC NAND Flash Memories Through Adaptive Data Refresh and Error Control Coding , 2014, J. Signal Process. Syst..

[52]  Cheng Li,et al.  Pannier: Design and Analysis of a Container-Based Flash Cache for Compound Objects , 2017, ACM Trans. Storage.

[53]  Ilya Krivorotov,et al.  Optical investigation of radiation induced conductivity changes in STT-RAM cells , 2016, 2016 Conference on Lasers and Electro-Optics (CLEO).

[54]  Ronald F. Demara,et al.  Read-Tuned STT-RAM and eDRAM Cache Hierarchies for Throughput and Energy Optimization , 2018, IEEE Access.

[55]  Renhai Chen,et al.  A reliability enhanced video storage architecture in hybrid SLC/MLC NAND flash memory , 2018, J. Syst. Archit..

[56]  John C. S. Lui,et al.  Analysis of Reliability Dynamics of SSD RAID , 2016, IEEE Transactions on Computers.

[57]  Hossein Asadi,et al.  Modeling Impact of Human Errors on the Data Unavailability and Data Loss of Storage Systems , 2018, IEEE Transactions on Reliability.

[58]  Hai Jin,et al.  A Case for Redundant Arrays of Inexpensive Disks (RAID) , 2002 .

[59]  Ronald F. DeMara,et al.  Energy-Efficient and Process-Variation-Resilient Write Circuit Schemes for Spin Hall Effect MRAM Device , 2017, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[60]  Albert Mo Kim Cheng,et al.  Disk failure prediction in heterogeneous environments , 2017, 2017 International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS).

[61]  Reza Salkhordeh,et al.  ReCA: An Efficient Reconfigurable Cache Architecture for Storage Systems with Online Workload Characterization , 2018, IEEE Transactions on Parallel and Distributed Systems.

[62]  Naehyuck Chang,et al.  Energy- and endurance-aware design of phase change memory caches , 2010, 2010 Design, Automation & Test in Europe Conference & Exhibition (DATE 2010).

[63]  Yiming Huai,et al.  Spin-Transfer Torque MRAM (STT-MRAM): Challenges and Prospects , 2008 .

[64]  J. Kessenich,et al.  Bit error rate in NAND Flash memories , 2008, 2008 IEEE International Reliability Physics Symposium.

[65]  Jongmoo Choi,et al.  Improving SSD reliability with RAID via Elastic Striping and Anywhere Parity , 2013, 2013 43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[66]  Arif Merchant,et al.  Flash Reliability in Production: The Expected and the Unexpected , 2016, FAST.

[67]  Deok-Hwan Kim,et al.  Reliability and performance enhancement technique for SSD array storage system using RAID mechanism , 2009, 2009 9th International Symposium on Communications and Information Technology.