Sorting in Parallel

Although more than a century passed before Menabrea's vision became a reality, today parallel computers with hundreds and even thousands of processors are used in a broad range of applications. One of the great challenges of parallel computing is the design of algorithms that make efficient use of the large number of processors in the machine. In this paper we explore parallel algorithms for one of the most fundamental problems in computer science: the problem of sorting a list of numbers. Specifically, given a list a1, a2, ... , an of integers, the sorting problem is that of finding a permutation of this list, 7T, such that a.(l) < a.(2) < ...< a(n) Before considering parallel sorting algorithms, we examine a sequential sorting algorithm; an algorithm for a computer with a single processor. One sequential algorithm for sorting is known as selection sort. The selection sort algorithm first examines each of the n integers a1, ... , an to find an integer ai with minimum value. Elements a1 and ai are then exchanged so that the element with minimum value is now in the first position in the list. Next, the algorithm examines each of the n 1 integers a2, ... ., an to find an integer with minimum value in this portion of the list. This integer is exchanged with a2 so that now the second smallest integer is in the second position in the list. In general, in the k th iteration the algorithm examines integers ak, .. , an, finds a minimum element in this list, and exchanges it with ak. At the end of nth iteration, the list is sorted. Observe that the first iteration of the algorithm requires n steps to find an element with minimum value and then some constant additional number of steps, c, to exchange this minimum value with a1. The next iteration requires n 1 steps to locate a minimum value in the remaining list and c steps to perform the exchange. In general, the kth iteration requires n k + 1 steps to locate a minimum value in the remaining list and c steps to perform the exchange. Since there are n iterations, the total number of steps or running time of the algorithm is