Fast parallel in-memory 64-bit sorting

Parallel in-memory 64-bit sorting is an important problem in Database Management Systems and other applications such as Internet Search Engines and Data Mining Tools. We propose a new algorithm that we call Parallel Counting Split Radix sort, PCS-Radix sort. The parallel stages of our algorithm increase the data locality, balance the load between processors caused by data skew and reduce significantly the amount of data communicated. The local stages of PCS-Radix sort are performed only on the bits of the key that have not been sorted during the parallel stages of the algorithm. All those improvements save a significant amount of computational and communication effort. Also, PCS-Radix sort adapts to any parallel computer by changing three simple algorithmic parameters. We have implemented our algorithm on a Cray T3E-900 and the results show that it is more than 2 times faster than the previous fastest 64-bit parallel sorting algorithm. PCS-Radix sort achieves a speed up of more than 23 in 32 processors in relation to the fastest sequential algorithm at our hands.