A Scalable Parallel Sorting Algorithm Using Exact Splitting

Sorting is one of the most fundamental algorithmic kernels, used by a large fraction of computer applications. This paper proposes a novel parallel sorting algorithm based on exact splitting that combines excellent scaling behavior with universal applicability. In contrast to many existing parallel sorting algorithms that make limiting assumptions regarding the input problem or the underlying computation model, our general-purpose algorithm can be used without restrictions on any MIMD-class computer architecture, demonstrating its full potential on massively parallel systems with distributed memory. It is comparison-based like most sequential sorting algorithms, handles an arbitrary number of keys per processing element, works in a deterministic way, does not fail in the presence of duplicate keys, minimizes the communication bandwidth requirements, does not require any knowledge of the key-value distribution, and uses only a small and a priori known amount of additional memory. Moreover, our algorithm can be turned into a stable sort without altering the time complexity, and can be made work in place. The total running time for sorting n elements on p processors is O( n log n + plog 2 n). Practical scalability is shown using more than thirty thousand compute nodes. This paper presents the first parallel sorting algorithm to combine all herein before mentioned properties, while laying the foundations to overcome scalability problems for sorting data on the next generation of massively parallel systems.