Reclaiming Memory for Lock-Free Data Structures: There has to be a Better Way

Memory reclamation for sequential or lock-based data structures is typically easy. However, memory reclamation for lock-free data structures is a significant challenge. Automatic techniques such as garbage collection are inefficient or use locks, and non-automatic techniques either have high overhead, or do not work for many reasonably simple data structures. For example, subtle problems can arise when hazard pointers, one of the most common non-automatic techniques, are applied to many natural lock-free data structures. Epoch based reclamation (EBR), which is by far the most efficient non-automatic technique, allows the number of unreclaimed objects to grow without bound, because one slow or crashed process can prevent all other processes from reclaiming memory. We develop a more efficient, distributed variant of EBR that solves this problem. It is based on signaling, which is provided by many operating systems, such as Linux and UNIX. Our new scheme takes O(1) amortized steps per high-level operation on the lock-free data structure and O(1) steps in the worst case each time an object is removed from the data structure. At any point, O(mn2) objects are waiting to be freed, where $n$ is the number of processes and m is a small constant for most data structures. Experiments show that our scheme has very low overhead: on average 10%, and at worst 28%, for a balanced binary search tree over many thread counts, operation mixes and contention levels. Our scheme also outperforms a highly efficient implementation of hazard pointers by an average of 75%. Typically, memory reclamation code is tightly woven into lock-free data structure code. To improve modularity and facilitate the comparison of different memory reclamation schemes, we also introduce a highly flexible abstraction. It allows a programmer to easily interchange schemes for reclamation, object pooling, allocation and deallocation with virtually no overhead, by changing a single line of code.

[1]  Faith Ellen,et al.  The amortized complexity of non-blocking binary search trees , 2014, PODC '14.

[2]  Maged M. Michael.  Hazard pointers: safe memory reclamation for lock-free objects , 2004, IEEE Transactions on Parallel and Distributed Systems.

[3]  Martin Schoeberl,et al.  Nonblocking real-time garbage collection , 2010, TECS.

[4]  Erez Petrank,et al.  Efficient Memory Management for Lock-Free Data Structures with Optimistic Access , 2015, SPAA.

[5]  Hyonho Lee.  Fast Local-Spin Abortable Mutual Exclusion with Bounded Space , 2010, OPODIS.

[6]  Maurice Herlihy,et al.  Fast non-intrusive memory reclamation for highly-concurrent data structures , 2016, ISMM.

[7]  Dan Alistarh,et al.  ThreadScan: Automatic and Scalable Memory Reclamation , 2015, SPAA.

[8]  Rafael Dueire Lins,et al.  Garbage collection: algorithms for automatic dynamic memory management , 1996 .

[9]  Faith Ellen,et al.  Non-blocking binary search trees , 2010, PODC.

[10]  Hagit Attiya,et al.  Concurrent updates with RCU: search tree as an example , 2014, PODC '14.

[11]  Erez Petrank,et al.  Drop the anchor: lightweight memory management for non-blocking data structures , 2013, SPAA.

[12]  Faith Ellen,et al.  A general technique for non-blocking trees , 2014, PPoPP '14.

[13]  Dan Alistarh,et al.  StackTrack: an automated transactional approach to concurrent memory reclamation , 2014, EuroSys '14.

[14]  Mark Moir,et al.  Lock-free reference counting , 2002, PODC '01.

[15]  Neeraj Mittal,et al.  A Fast Lock-Free Internal Binary Search Tree , 2015, ICDCN.

[16]  Neeraj Mittal,et al.  Fast concurrent lock-free binary search trees , 2014, PPoPP.

[17]  Rachid Guerraoui,et al.  Fast and Robust Memory Reclamation for Concurrent Data Structures , 2016, SPAA.

[18]  Niloufar Shafiei.  Non-blocking Patricia Tries with Replace Operations , 2013, 2013 IEEE 33rd International Conference on Distributed Computing Systems.

[19]  Maurice Herlihy,et al.  Nonblocking memory management support for dynamic-sized data structures , 2005, TOCS.

[20]  Trevor Brown,et al.  Non-blocking k-ary Search Trees , 2011, OPODIS.

[21]  Maurice Herlihy,et al.  On the power of hardware transactional memory to simplify memory management , 2011, PODC '11.

[22]  Eran Yahav,et al.  Practical concurrent binary search trees via logical ordering , 2014, PPoPP '14.

[23]  Marina Papatriantafilou,et al.  Efficient and Reliable Lock-Free Memory Reclamation Based on Reference Counting , 2009, IEEE Transactions on Parallel and Distributed Systems.

[24]  Keir Fraser,et al.  Practical lock-freedom , 2003 .

[25]  Jonathan Walpole,et al.  Performance of memory reclamation for lockless synchronization , 2007, J. Parallel Distributed Comput..

[26]  Maurice Herlihy,et al.  A methodology for implementing highly concurrent data objects , 1993, TOPL.

[27]  Erez Petrank,et al.  Automatic memory reclamation for lock-free data structures , 2015, OOPSLA.

[28]  Maurice Herlihy,et al.  A Lazy Concurrent List-Based Set Algorithm , 2007, Parallel Process. Lett..

[29]  Keir Fraser,et al.  A Practical Multi-word Compare-and-Swap Operation , 2002, DISC.

[30]  Paul E. McKenney,et al.  READ-COPY UPDATE: USING EXECUTION HISTORY TO SOLVE CONCURRENCY PROBLEMS , 2002 .

[31]  Wojciech M. Golab,et al.  Making objects writable , 2014, PODC '14.

[32]  Shane V. Howley,et al.  A non-blocking internal binary search tree , 2012, SPAA '12.