SoaAlloc: A Lock-free Hierarchical Bitmap-based Object Allocator for GPUs

Designing dynamic memory allocators for GPUs is challenging because applications can issue allocation requests in a highly parallel fashion and memory access and the data layout must be optimized to achieve good memory bandwidth utilization. Despite recent advances in GPU computing, current memory allocators for SIMD architectures are still not suitable for structured data because they fail to incorporate well-known best practices for optimizing memory access. Therefore, we developed SoaAlloc, a new dynamic object allocator for GPUs. Besides delivering competitive raw (de)allocation performance, SoaAlloc improves the usage of allocated memory with a Structure of Arrays (SOA) data layout and achieves low memory fragmentation through efficient management of free and allocated memory blocks with lock-free, hierarchical bitmaps. The SOA layout alone results in a 2x speedup of application code over state-of-the-art allocators in our benchmarks. Furthermore, SoaAlloc is the first GPU object allocator that provides a do-all operation, which is an important recurring pattern in high-performance code where parallelism is expressed over a set of objects.