NVTraverse: in NVRAM data structures, the destination is more important than the journey

The recent availability of fast, dense, byte-addressable non-volatile memory has led to increasing interest in the problem of designing durable data structures that can recover from system crashes. However, designing durable concurrent data structures that are correct and efficient has proven to be very difficult, leading to many inefficient or incorrect algorithms. In this paper, we present a general transformation that takes a lock-free data structure from a general class called traversal data structure (that we formally define) and automatically transforms it into an implementation of the data structure for the NVRAM setting that is provably durably linearizable and highly efficient. The transformation hinges on the observation that many data structure operations begin with a traversal phase that does not need to be persisted, and thus we only begin persisting when the traversal reaches its destination. We demonstrate the transformation's efficiency through extensive measurements on a system with Intel's recently released Optane DC persistent memory, showing that it can outperform competitors on many workloads.

[1]  Maurice Herlihy,et al.  A Simple Optimistic Skiplist Algorithm , 2007, SIROCCO.

[2]  Rajesh K. Gupta,et al.  NV-Heaps: making persistent objects fast and safe with next-generation, non-volatile memories , 2011, ASPLOS XVI.

[3]  Erez Petrank,et al.  A practical wait-free simulation for lock-free data structures , 2014, PPoPP '14.

[4]  Trevor Brown A Template for Implementing Fast Lock-free Trees Using HTM , 2017, PODC.

[5]  Guy E. Blelloch,et al.  The Parallel Persistent Memory Model , 2018, SPAA.

[6]  Onur Mutlu,et al.  NVMOVE: Helping Programmers Move to Byte-Based Persistence , 2016, INFLOW@OSDI.

[7]  Faith Ellen,et al.  Non-blocking binary search trees , 2010, PODC.

[8]  Viktor Vafeiadis,et al.  Persistence semantics for weak memory: integrating epoch persistency with the TSO memory model , 2018, Proc. ACM Program. Lang..

[9]  Qin Jin,et al.  Persistent B+-Trees in Non-Volatile Main Memory , 2015, Proc. VLDB Endow..

[10]  Margo I. Seltzer,et al.  Persistent Memory Transactions , 2018, ArXiv.

[11]  Roy H. Campbell,et al.  Consistent and Durable Data Structures for Non-Volatile Byte-Addressable Memory , 2011, FAST.

[12]  Sam H. Noh,et al.  WORT: Write Optimal Radix Tree for Persistent Memory Storage Systems , 2017, FAST.

[13]  Rachid Guerraoui,et al.  Robust emulations of shared memory in a crash-recovery model , 2004, 24th International Conference on Distributed Computing Systems, 2004. Proceedings..

[14]  Danny Hendler,et al.  Nesting-Safe Recoverable Linearizability: Modular Constructions for Non-Volatile Memory , 2018, PODC.

[15]  Maurice Herlihy,et al.  A persistent lock-free queue for non-volatile memory , 2018, PPoPP.

[16]  Adam Silberstein,et al.  Benchmarking cloud serving systems with YCSB , 2010, SoCC '10.

[17]  Timothy L. Harris,et al.  A Pragmatic Implementation of Non-blocking Linked-Lists , 2001, DISC.

[18]  Guy E. Blelloch,et al.  Delay-Free Concurrency on Faulty Persistent Memory , 2018, SPAA.

[19]  Michael L. Scott,et al.  iDO: Compiler-Directed Failure Atomicity for Nonvolatile Memory , 2018, 2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[20]  Tudor David,et al.  Log-Free Concurrent Data Structures , 2018, USENIX Annual Technical Conference.

[21]  Weimin Zheng,et al.  DudeTM: Building Durable Transactions with Decoupling for Persistent Memory , 2017, ASPLOS.

[22]  James R. Larus,et al.  Efficient logging in non-volatile memory by exploiting coherency protocols , 2017, Proc. ACM Program. Lang..

[23]  Pascal Felber,et al.  Romulus: Efficient Algorithms for Persistent Transactional Memory , 2018, SPAA.

[24]  Taesoo Kim,et al.  Recipe: converting concurrent DRAM indexes to persistent-memory indexes , 2019, SOSP.

[25]  Maurice Herlihy,et al.  Linearizability: a correctness condition for concurrent objects , 1990, TOPL.

[26]  Jian Xu,et al.  NOVA: A Log-structured File System for Hybrid Volatile/Non-volatile Main Memories , 2016, FAST.

[27]  Erez Petrank,et al.  LOFT: lock-free transactional data structures , 2019, PPoPP.

[28]  Thomas F. Wenisch,et al.  High-Performance Transactions for Persistent Memories , 2016, ASPLOS.

[29]  Wojciech M. Golab,et al.  Robust Shared Objects for Non-Volatile Main Memory , 2015, OPODIS.

[30]  Bingsheng He,et al.  NV-Tree: Reducing Consistency Cost for NVM-based Single Level Systems , 2015, FAST.

[31]  Viktor Vafeiadis,et al.  Persistency semantics of the Intel-x86 architecture , 2019, Proc. ACM Program. Lang..

[32]  Yujie Liu,et al.  Optimizing Persistent Memory Transactions , 2019, 2019 28th International Conference on Parallel Architectures and Compilation Techniques (PACT).

[33]  Ryan Johnson,et al.  Scalable Logging through Emerging Non-Volatile Memory , 2014, Proc. VLDB Endow..

[34]  Rachid Guerraoui,et al.  The Inherent Cost of Remembering Consistently , 2018, SPAA.

[35]  Tudor David,et al.  Asynchronized Concurrency: The Secret to Scaling Concurrent Search Data Structures , 2015, ASPLOS.

[36]  Laurent Amsaleg,et al.  NV-Tree: An Efficient Disk-Based Index for Approximate Search in Very Large High-Dimensional Collections , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Maged M. Michael Safe memory reclamation for dynamic lock-free objects using atomic reads and writes , 2002, PODC '02.

[38]  Neeraj Mittal,et al.  Fast concurrent lock-free binary search trees , 2014, PPoPP.

[39]  Maged M. Michael,et al.  Simple, fast, and practical non-blocking and blocking concurrent queue algorithms , 1996, PODC '96.

[40]  Marcos K. Aguilera,et al.  Strict Linearizability and the Power of Aborting , 2003 .

[41]  Andy Rudoff,et al.  Persistent Memory Programming , 2017, login Usenix Mag..

[42]  Michael M. Swift,et al.  Mnemosyne: lightweight persistent memory , 2011, ASPLOS XVI.

[43]  Faith Ellen,et al.  A general technique for non-blocking trees , 2014, PPoPP '14.

[44]  Viktor Vafeiadis,et al.  Weak persistency semantics from the ground up: formalising the persistency semantics of ARMv8 and transactional models , 2019, Proc. ACM Program. Lang..

[45]  Josep Torrellas,et al.  AutoPersist: an easy-to-use Java NVM framework based on reachability , 2019, PLDI.

[46]  Michael L. Scott,et al.  Linearizability of Persistent Memory Objects Under a Full-System-Crash Failure Model , 2016, DISC.

[47]  Pascal Felber,et al.  OneFile: A Wait-Free Persistent Transactional Memory , 2019, 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[48]  Hans-Juergen Boehm,et al.  Atlas: leveraging locks for non-volatile memory consistency , 2014, OOPSLA.

[49]  Jie Ren,et al.  Architecture-Aware, High Performance Transaction for Persistent Memory , 2019, ArXiv.