A Survey of Software Techniques for Using Non-Volatile Memories for Storage and Main Memory Systems

Non-volatile memory (NVM) devices, such as Flash, phase change RAM, spin transfer torque RAM, and resistive RAM, offer several advantages and challenges when compared to conventional memory technologies, such as DRAM and magnetic hard disk drives (HDDs). In this paper, we present a survey of software techniques that have been proposed to exploit the advantages and mitigate the disadvantages of NVMs when used for designing memory systems, and, in particular, secondary storage (e.g., solid state drive) and main memory. We classify these software techniques along several dimensions to highlight their similarities and differences. Given that NVMs are growing in popularity, we believe that this survey will motivate further research in the field of software technology for NVMs.

[1]  Liang Shi,et al.  Error Model Guided Joint Performance and Endurance Optimization for Flash Memory , 2014, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[2]  Ke Zhou,et al.  FlexECC: Partially Relaxing ECC of MLC SSD for Better Cache Performance , 2014, USENIX Annual Technical Conference.

[3]  Dong Li,et al.  DESTINY: A tool for modeling emerging 3D NVM and eDRAM caches , 2015, 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[4]  Somayeh Sardashti,et al.  UniFI: leveraging non-volatile memories for a unified fault tolerance and idle power management technique , 2012, ICS '12.

[5]  Li-Pin Chang,et al.  A low-cost wear-leveling algorithm for block-mapping solid-state disks , 2011, LCTES '11.

[6]  Rajesh K. Gupta,et al.  NV-Heaps: making persistent objects fast and safe with next-generation, non-volatile memories , 2011, ASPLOS XVI.

[7]  Karin Strauss,et al.  Using managed runtime systems to tolerate holes in wearable memories , 2013, PLDI 2013.

[8]  Suman Nath,et al.  Rethinking Database Algorithms for Phase Change Memory , 2011, CIDR.

[9]  Osman S. Unsal,et al.  Flash correct-and-refresh: Retention-aware error management for increased flash memory lifetime , 2012, 2012 IEEE 30th International Conference on Computer Design (ICCD).

[10]  Maya Gokhale,et al.  DI-MMAP—a scalable memory-map runtime for out-of-core data-intensive applications , 2015, Cluster Computing.

[11]  Kai Shen,et al.  A performance evaluation of scientific I/O workloads on Flash-based SSDs , 2009, 2009 IEEE International Conference on Cluster Computing and Workshops.

[12]  Nisha Talagala,et al.  HEC: improving endurance of high performance flash-based cache devices , 2013, SYSTOR '13.

[13]  Anand Kulkarni,et al.  nand Flash Memory and Its Role in Storage Architectures , 2008, Proceedings of the IEEE.

[14]  Antony I. T. Rowstron,et al.  Migrating server storage to SSDs: analysis of tradeoffs , 2009, EuroSys '09.

[15]  Ren-Shuo Liu,et al.  NVM duet: unified working memory and persistent store architecture , 2014, ASPLOS.

[16]  Andrea C. Arpaci-Dusseau,et al.  De-indirection for flash-based SSDs with nameless writes , 2012, FAST.

[17]  Orion Hodson,et al.  Whole-system persistence , 2012, ASPLOS XVII.

[18]  Peter J. Varman,et al.  Bridging the programming gap between persistent and volatile memory using WrAP , 2013, CF '13.

[19]  Chao Wang,et al.  NVMalloc: Exposing an Aggregate SSD Store as a Memory Partition in Extreme-Scale Machines , 2012, 2012 IEEE 26th International Parallel and Distributed Processing Symposium.

[20]  Wei Wu,et al.  Optimizing NAND flash-based SSDs via retention relaxation , 2012, FAST.

[21]  Osman S. Unsal,et al.  Neighbor-cell assisted error correction for MLC NAND flash memories , 2014, SIGMETRICS '14.

[22]  Kevin Fu,et al.  Mementos: system support for long-running computation on RFID-scale devices , 2011, ASPLOS XVI.

[23]  Ren-Shuo Liu,et al.  DuraCache: A durable SSD cache using MLC NAND flash , 2013, 2013 50th ACM/EDAC/IEEE Design Automation Conference (DAC).

[24]  Paolo Faraboschi,et al.  Operating System Support for NVM+DRAM Hybrid Main Memory , 2009, HotOS.

[25]  Chanik Park,et al.  Active disk meets flash: a case for intelligent SSDs , 2013, ICS '13.

[26]  Jacob Nelson,et al.  Approximate storage in solid-state memories , 2013, 2013 46th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[27]  Yuan Xie,et al.  Leveraging 3D PCRAM technologies to reduce checkpoint overhead for future exascale systems , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[28]  Tong Zhang,et al.  Over-clocked SSD: Safely running beyond flash memory chip I/O clock specs , 2014, 2014 IEEE 20th International Symposium on High Performance Computer Architecture (HPCA).

[29]  Onur Mutlu,et al.  Program interference in MLC NAND flash memory: Characterization, modeling, and mitigation , 2013, ICCD.

[30]  Yuan Xie,et al.  Kiln: Closing the performance gap between systems with and without persistence support , 2013, 2013 46th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[31]  Dejan S. Milojicic,et al.  Optimizing Checkpoints Using NVM as Virtual Memory , 2013, 2013 IEEE 27th International Symposium on Parallel and Distributed Processing.

[32]  Rina Panigrahy,et al.  Design Tradeoffs for SSD Performance , 2008, USENIX Annual Technical Conference.

[33]  Alessandro Curioni,et al.  Rebasing I/O for Scientific Computing: Leveraging Storage Class Memory in an IBM BlueGene/Q Supercomputer , 2014, ISC.

[34]  Dongkun Shin,et al.  Differentiated space allocation for wear leveling on phase-change memory-based storage device , 2014, IEEE Transactions on Consumer Electronics.

[35]  Hua Wang,et al.  A novel I/O scheduler for SSD with improved performance and lifetime , 2013, 2013 IEEE 29th Symposium on Mass Storage Systems and Technologies (MSST).

[36]  Hans-Juergen Boehm,et al.  Atlas: leveraging locks for non-volatile memory consistency , 2014, OOPSLA.

[37]  Hyojun Kim,et al.  BPLRU: A Buffer Management Scheme for Improving Random Writes in Flash Storage , 2008, FAST.

[38]  Stratis Viglas,et al.  Flashing up the storage layer , 2008, Proc. VLDB Endow..

[39]  P. Mohapatra,et al.  Video Delivery Challenges and Opportunities in 4G Networks , 2013, IEEE MultiMedia.

[40]  Anand Sivasubramaniam,et al.  HybridStore: A Cost-Efficient, High-Performance Storage System Combining SSDs and HDDs , 2011, 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems.

[41]  Peter Desnoyers,et al.  Write Endurance in Flash Drives: Measurements and Analysis , 2010, FAST.

[42]  Michael Wu,et al.  eNVy: a non-volatile, main memory storage system , 1994, ASPLOS VI.

[43]  Chaitali Chakrabarti,et al.  Energy-aware error control coding for Flash memories , 2009, 2009 46th ACM/IEEE Design Automation Conference.

[44]  Chundong Wang,et al.  Observational wear leveling: An efficient algorithm for flash memory management , 2012, DAC Design Automation Conference 2012.

[45]  Duane Mills,et al.  19.7 A 16Gb ReRAM with 200MB/s write and 1GB/s read in 27nm technology , 2014, 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC).

[46]  Jongmoo Choi,et al.  Caching less for better performance: balancing cache size and update cost of flash memory cache in hybrid storage systems , 2012, FAST.

[47]  Bruce Jacob,et al.  The performance of PC solid-state disks (SSDs) as a function of bandwidth, concurrency, device architecture, and system organization , 2009, ISCA '09.

[48]  Youngjae Kim,et al.  DFTL: a flash translation layer employing demand-based selective caching of page-level address mappings , 2009, ASPLOS.

[49]  C. Kirsch.  Combo Drive : Optimizing Cost and Performance in a Heterogeneous Storage Device , 2009 .

[50]  Jeffrey S. Vetter,et al.  A Survey of Techniques for Modeling and Improving Reliability of Computing Systems , 2016, IEEE Transactions on Parallel and Distributed Systems.

[51]  Steven Swanson,et al.  The bleak future of NAND flash memory , 2012, FAST.

[52]  Arif Merchant,et al.  Janus: Optimal Flash Provisioning for Cloud Storage Workloads , 2013, USENIX Annual Technical Conference.

[53]  Onur Mutlu,et al.  Error patterns in MLC NAND flash memory: Measurement, characterization, and analysis , 2012, 2012 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[54]  John Shalf,et al.  Triple-A: a Non-SSD based autonomic all-flash array for high performance storage systems , 2014, ASPLOS.

[55]  Mithuna Thottethodi,et al.  SieveStore: a highly-selective, ensemble-level disk cache for cost-performance , 2010, ISCA '10.

[56]  Nikil D. Dutt,et al.  Meta-Cure: A reliability enhancement strategy for metadata in NAND flash memory storage systems , 2012, DAC Design Automation Conference 2012.

[57]  Jianhua Li,et al.  Hybrid nonvolatile disk cache for energy-efficient and high-performance systems , 2013, TODE.

[58]  Onur Mutlu,et al.  A Case for Effic ient Hardware/Soft ware Cooperative Management of Storage and Memory , 2013 .

[59]  Haibo Chen,et al.  Persistent Transactional Memory , 2015, IEEE Computer Architecture Letters.

[60]  Luis Ceze,et al.  Exploring storage class memory with key value stores , 2013, INFLOW '13.

[61]  Tei-Wei Kuo,et al.  Endurance Enhancement of Flash-Memory Storage, Systems: An Efficient Static Wear Leveling Design , 2007, 2007 44th ACM/IEEE Design Automation Conference.

[62]  Vivek S. Pai,et al.  SSDAlloc: Hybrid SSD/RAM Memory Management Made Easy , 2011, NSDI.

[63]  Song Jiang,et al.  Synergistic coupling of SSD and hard disk for QoS-aware virtual memory , 2013, 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).

[64]  Norman P. Jouppi,et al.  FREE-p: Protecting non-volatile memory against both hard and soft errors , 2011, 2011 IEEE 17th International Symposium on High Performance Computer Architecture.

[65]  Hsien-Hsin S. Lee,et al.  SAFER: Stuck-At-Fault Error Recovery for Memories , 2010, 2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture.

[66]  Xiaodong Zhang,et al.  Essential roles of exploiting internal parallelism of flash memory based solid state drives in high-speed data processing , 2011, 2011 IEEE 17th International Symposium on High Performance Computer Architecture.

[67]  Rami G. Melhem,et al.  Data Dependent Sparing to Manage Better-Than-Bad Blocks , 2013, IEEE Computer Architecture Letters.

[68]  Vijayalakshmi Srinivasan,et al.  Efficient scrub mechanisms for error-prone emerging memories , 2012, IEEE International Symposium on High-Performance Comp Architecture.

[69]  Parthasarathy Ranganathan,et al.  Consistent, durable, and safe memory management for byte-addressable non volatile main memory , 2013, TRIOS@SOSP.

[70]  Jin-Soo Kim,et al.  BEST: Best-effort energy saving techniques for NAND flash-based hybrid storage , 2012, IEEE Transactions on Consumer Electronics.

[71]  Chundong Wang,et al.  SAW: System-assisted wear leveling on the write endurance of NAND flash devices , 2013, 2013 50th ACM/EDAC/IEEE Design Automation Conference (DAC).

[72]  Li-Pin Chang,et al.  On efficient wear leveling for large-scale flash-memory storage systems , 2007, SAC '07.

[73]  Hua Wang,et al.  Improve Effective Capacity and Lifetime of Solid State Drives , 2013, 2013 IEEE Eighth International Conference on Networking, Architecture and Storage.

[74]  Arun Jagatheesan,et al.  Understanding the Impact of Emerging Non-Volatile Memories on High-Performance, IO-Intensive Computing , 2010, 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis.

[75]  Luis A. Lastras,et al.  PreSET: Improving performance of phase change memories by exploiting asymmetry in write times , 2012, 2012 39th Annual International Symposium on Computer Architecture (ISCA).

[76]  Sungjin Lee,et al.  Lifetime improvement of NAND flash-based storage systems using dynamic program and erase scaling , 2014, FAST.

[77]  Yiran Chen,et al.  A Hybrid solid-state storage architecture for the performance, energy consumption, and lifetime improvement , 2010, HPCA - 16 2010 The Sixteenth International Symposium on High-Performance Computer Architecture.

[78]  Tong Zhang,et al.  Quasi-nonvolatile SSD: Trading flash memory nonvolatility to improve storage system performance for enterprise applications , 2012, IEEE International Symposium on High-Performance Comp Architecture.

[79]  Dong Li,et al.  Identifying Opportunities for Byte-Addressable Non-Volatile Memory in Extreme-Scale Scientific Applications , 2012, 2012 IEEE 26th International Parallel and Distributed Processing Symposium.

[80]  Vijayalakshmi Srinivasan,et al.  Scalable high performance main memory system using phase-change memory technology , 2009, ISCA '09.

[81]  Tian Luo,et al.  CAFTL: A Content-Aware Flash Translation Layer Enhancing the Lifespan of Flash Memory based Solid State Drives , 2011, FAST.

[82]  Jongman Kim,et al.  A semi-preemptive garbage collector for solid state drives , 2011, (IEEE ISPASS) IEEE INTERNATIONAL SYMPOSIUM ON PERFORMANCE ANALYSIS OF SYSTEMS AND SOFTWARE.

[83]  Piero Olivo,et al.  Flash memory cells-an overview , 1997, Proc. IEEE.

[84]  Christopher Frost,et al.  Better I/O through byte-addressable, persistent memory , 2009, SOSP '09.

[85]  Manish Parashar,et al.  Exploring energy and performance behaviors of data-intensive scientific workflows on systems with deep memory hierarchies , 2013, 20th Annual International Conference on High Performance Computing.

[86]  David Blaauw,et al.  Exploring DRAM organizations for energy-efficient and resilient exascale memories , 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[87]  Michael M. Swift,et al.  FlashTier: a lightweight, consistent and durable storage cache , 2012, EuroSys '12.

[88]  Wei-Che Tseng,et al.  Write activity reduction on flash main memory via smart victim cache , 2010, GLSVLSI '10.

[89]  Tong Zhang,et al.  OFWAR: Reducing SSD Response Time Using On-Demand Fast-Write-and-Rewrite , 2014, IEEE Transactions on Computers.

[90]  Paul H. Siegel,et al.  Characterizing flash memory: Anomalies, observations, and applications , 2009, 2009 42nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[91]  Ming Zhao,et al.  Write policies for host-side flash caches , 2013, FAST.

[92]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[93]  Trevor N. Mudge,et al.  Improving NAND Flash Based Disk Caches , 2008, 2008 International Symposium on Computer Architecture.

[94]  Sungjin Lee,et al.  Refactored Design of I/O Architecture for Flash Storage , 2015, IEEE Computer Architecture Letters.

[95]  Dutch T. Meyer,et al.  Strata: scalable high-performance storage on virtualized non-volatile memory , 2014, FAST.

[96]  Roy H. Campbell,et al.  Consistent and Durable Data Structures for Non-Volatile Byte-Addressable Memory , 2011, FAST.

[97]  A. L. Narasimha Reddy,et al.  Exploiting Concurrency to Improve Latency and throughput in a Hybrid Storage System , 2010, 2010 IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[98]  Hyunjun Kim,et al.  In-memory file system for non-volatile memory , 2013, RACS.

[99]  Sanjay Kumar,et al.  System software for persistent memory , 2014, EuroSys '14.

[100]  Mahesh Balakrishnan,et al.  Extending SSD Lifetimes with Disk-Based Write Caches , 2010, FAST.

[101]  Chundong Wang,et al.  Extending the lifetime of NAND flash memory by salvaging bad blocks , 2012, 2012 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[102]  Dong Li,et al.  PCM-Based Durable Write Cache for Fast Disk I/O , 2012, 2012 IEEE 20th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[103]  Ada Gavrilovska,et al.  Reducing the cost of persistence for nonvolatile heaps in end user devices , 2014, 2014 IEEE 20th International Symposium on High Performance Computer Architecture (HPCA).

[104]  Feng Chen,et al.  Hystor: making the best use of solid state drives in high performance storage systems , 2011, ICS '11.

[105]  Michael M. Swift,et al.  Mnemosyne: lightweight persistent memory , 2011, ASPLOS XVI.

[106]  Karin Strauss,et al.  Use ECP, not ECC, for hard failures in resistive memories , 2010, ISCA.

[107]  Sungjin Lee,et al.  Lifetime management of flash-based SSDs using recovery-aware dynamic throttling , 2012, FAST.

[108]  Jeanna Matthews,et al.  Intel® Turbo Memory: Nonvolatile disk caches in the storage hierarchy of mainstream computer systems , 2008, TOS.

[109]  Rajesh K. Gupta,et al.  Onyx: A Prototype Phase Change Memory Storage Array , 2011, HotStorage.

[110]  Xubin He,et al.  Reducing SSD read latency via NAND flash program and erase suspension , 2012, FAST.

[111]  Huawei Li,et al.  SoftPCM: Enhancing Energy Efficiency and Lifetime of Phase Change Memory in Video Applications via Approximate Write , 2012, 2012 IEEE 21st Asian Test Symposium.

[112]  Rajesh K. Gupta,et al.  Moneta: A High-Performance Storage Array Architecture for Next-Generation, Non-volatile Memories , 2010, 2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture.

[113]  Eunji Lee,et al.  Shortcut-JFS: A write efficient journaling file system for phase change memory , 2012, 012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST).

[114]  A. L. Narasimha Reddy,et al.  SCMFS: A file system for Storage Class Memory , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[115]  Hyojun Kim,et al.  Evaluating Phase Change Memory for Enterprise Storage Systems: A Study of Caching and Tiering Approaches , 2014, TOS.

[116]  Sangyeun Cho,et al.  Memorage: emerging persistent RAM based malleable main memory and storage architecture , 2013, ICS '13.

[117]  Jeffrey S. Vetter,et al.  Opportunities for Nonvolatile Memory Systems in Extreme-Scale High-Performance Computing , 2015, Computing in Science & Engineering.

[118]  Robert B. Ross,et al.  On the role of burst buffers in leadership-class storage systems , 2012, 012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST).

[119]  Kevin Marquet,et al.  Incremental checkpointing of program state to NVRAM for transiently-powered systems , 2014, 2014 9th International Symposium on Reconfigurable and Communication-Centric Systems-on-Chip (ReCoSoC).

[120]  H. Howie Huang,et al.  RePRAM: Re-cycling PRAM faulty blocks for extended lifetime , 2012, IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012).

[121]  Thomas F. Wenisch,et al.  Memory persistency , 2014, 2014 ACM/IEEE 41st International Symposium on Computer Architecture (ISCA).

[122]  M.H. Kryder,et al.  After Hard Drives—What Comes Next? , 2009, IEEE Transactions on Magnetics.

[123]  Sang-Won Lee,et al.  Design of flash-based DBMS: an in-page logging approach , 2007, SIGMOD '07.

[124]  Jeffrey S. Vetter,et al.  A Survey of Methods for Analyzing and Improving GPU Energy Efficiency , 2014, ACM Comput. Surv..

[125]  Sanghyuk Jung,et al.  A process-aware hot/cold identification scheme for flash memory storage systems , 2010, IEEE Transactions on Consumer Electronics.

[126]  Xiaodong Zhang,et al.  Understanding intrinsic characteristics and system implications of flash memory based solid state drives , 2009, SIGMETRICS '09.

[127]  Sparsh Mittal,et al.  A survey of power management techniques for phase change memory , 2016, Int. J. Comput. Aided Eng. Technol..

[128]  Li Li,et al.  Adaptive Wear-Leveling in Flash-Based Memory , 2015, IEEE Computer Architecture Letters.

[129]  Qing Yang,et al.  I-CASH: Intelligently Coupled Array of SSD and HDD , 2011, 2011 IEEE 17th International Symposium on High Performance Computer Architecture.

[130]  Engin Ipek,et al.  Dynamically replicated memory: building reliable systems from nanoscale resistive memories , 2010, ASPLOS XV.

[131]  Dong Li,et al.  A Survey Of Architectural Approaches for Managing Embedded DRAM and Non-Volatile On-Chip Caches , 2015, IEEE Transactions on Parallel and Distributed Systems.

[132]  Joonwon Lee,et al.  A comprehensive study of energy efficiency and performance of flash-based SSD , 2011, J. Syst. Archit..

[133]  Vanish Talwar,et al.  Using active NVRAM for I/O staging , 2011, PDAC '11.