Ranomized Parallel Computation

The subject of this thesis is randomized parallel algorithms. The goal is to demonstrate the power of randomization in designing efficient parallel algorithms for various computational problems. The specific problems we are concerned with are: sorting, routing, and combinatorial optimization. A survey of existing randomized parallel algorithms for many important problems is given first. This is followed by a procedure for deriving randomized parallel algorithms for selection and sorting. Next we present optimal algorithms for sorting both general and integer keys. We also present algorithms that run in sub-logarithmic time. A very important problem in parallel computation is routing. We give an optimal algorithm for permutation routing on a square mesh. We also identify a class of mesh-like networks that have optimal diameter. Many heuristic algorithms have been proposed for solving presumably hard combinatorial problems (like the travelling salesman problem etc.). Simulated Annealing is one such heuristic that employs randomization. We modify this procedure for problems that have certain separation properties. We prove this modified procedure (called Nested Annealing) has a worst case performance better than that of Simulated Annealing. Nested Annealing is shown to be easily parallelizable.

[1]  E. Reingold,et al.  Combinatorial Algorithms: Theory and Practice , 1977 .

[2]  Donald E. Knuth,et al.  The art of computer programming: sorting and searching (volume 3) , 1973 .

[3]  Leslie G. Valiant,et al.  Fast probabilistic algorithms for hamiltonian circuits and matchings , 1977, STOC '77.

[4]  J. Darroch.  On the Distribution of the Number of Successes in Independent Trials , 1964 .

[5]  Alfred V. Aho,et al.  The Design and Analysis of Computer Algorithms , 1974 .

[6]  Yiming Ma,et al.  The distance bound for sorting on mesh-connected processor arrays is tight , 1986, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[7]  Adi Shamir,et al.  An optimal sorting algorithm for mesh connected computers , 1986, STOC '86.

[8]  Robert E. Tarjan,et al.  Depth-First Search and Linear Graph Algorithms , 1972, SIAM J. Comput..

[9]  J. Hopcroft,et al.  Efficient algorithms for graph manipulation , 1971 .

[10]  Ronald L. Rivest,et al.  Expected time bounds for selection , 1975, Commun. ACM.

[11]  Leslie G. Valiant,et al.  A logarithmic time sort for linear size networks , 1982, STOC.

[12]  E. Szemerédi,et al.  O(n LOG n) SORTING NETWORK. , 1983 .

[13]  Rüdiger Reischuk,et al.  A fast probabilistic parallel sorting algorithm , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[14]  Rusins Freivalds,et al.  Fast Probabilistic Algorithms , 1979, MFCS.

[15]  Manfred Kunde.  Optimal Sorting on Multi-Dimensionally Mesh-Connected Computers , 1987, STACS.

[16]  Thompson.  The VLSI Complexity of Sorting , 1983, IEEE Transactions on Computers.

[17]  Sartaj Sahni,et al.  Data broadcasting in SIMD computers , 1981, IEEE Transactions on Computers.

[18]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[19]  Manoj Kumar,et al.  An Efficient Implementation of Batcher's Odd-Even Merge Algorithm and Its Application in Parallel Sorting Schemes , 1983, IEEE Transactions on Computers.

[20]  Richard Cole,et al.  Approximate and exact parallel scheduling with applications to list, tree and graph problems , 1986, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[21]  Victor Y. Pan,et al.  Efficient parallel solution of linear systems , 1985, STOC '85.

[22]  Marvin C. Paull,et al.  On sorting networks , 1972 .

[23]  Richard Cole,et al.  Parallel merge sort , 1988, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[24]  Yoshihide Igarashi,et al.  Some Parallel Sorts on a Mesh-Connected Processor Array and Their Time Efficiency , 1986, J. Parallel Distributed Comput..

[25]  Richard P. Brent,et al.  The Parallel Evaluation of General Arithmetic Expressions , 1974, JACM.

[26]  G. P. Bhattacharjee,et al.  A parallel search algorithm for directed acyclic graphs , 1984, BIT.

[27]  F. Fich.  Two problems in concrete complexity: cycle detection and parallel prefix computation , 1982 .

[28]  Steven Fortune,et al.  Parallelism in random access machines , 1978, STOC.

[29]  Denise Marie Eckstein.  Parallel graph processing using depth-first search and breadth-first search. , 1977 .

[30]  Volker Strassen,et al.  A Fast Monte-Carlo Test for Primality , 1977, SIAM J. Comput..

[31]  Kurt Mehlhorn,et al.  Data Structures and Algorithms 1: Sorting and Searching , 2011, EATCS Monographs on Theoretical Computer Science.

[32]  Leslie G. Valiant,et al.  A Scheme for Fast Parallel Communication , 1982, SIAM J. Comput..

[33]  R. Tarjan,et al.  A Separator Theorem for Planar Graphs , 1977 .

[34]  János Komlós,et al.  An 0(n log n) sorting network , 1983, STOC.

[35]  E. Szemerédi,et al.  Sorting inc logn parallel steps , 1983 .

[36]  Johan Håstad,et al.  Optimal bounds for decision problems on the CRCW PRAM , 1987, STOC.

[37]  Richard M. Karp,et al.  A fast parallel algorithm for the maximal independent set problem , 1984, STOC '84.

[38]  Larry Carter,et al.  Universal Classes of Hash Functions , 1979, J. Comput. Syst. Sci..

[39]  Sanguthevar Rajasekaran,et al.  Derivation of Randomized Algorithms. , 1985 .

[40]  J. Doug Tygar,et al.  Efficient Parallel Pseudo-Random Number Generation , 1985, CRYPTO.

[41]  Leslie G. Valiant,et al.  Parallelism in Comparison Problems , 1975, SIAM J. Comput..

[42]  Rüdiger Reischuk.  Probabilistic Parallel Algorithms for Sorting and Selection , 1985, SIAM J. Comput..

[43]  Allan Borodin,et al.  Routing, merging and sorting on parallel models of computation , 1982, STOC '82.

[44]  Kenneth E. Batcher,et al.  Sorting networks and their applications , 1968, AFIPS Spring Joint Computing Conference.

[45]  Justin R. Smith.  Parallel Algorithms for Depth-First Searches I. Planar Graphs , 1986, SIAM J. Comput..

[46]  Uzi Vishkin,et al.  Finding the Maximum, Merging, and Sorting in a Parallel Computation Model , 1981, J. Algorithms.

[47]  Leslie G. Valiant,et al.  Universal schemes for parallel communication , 1981, STOC '81.

[48]  Dexter Kozen,et al.  Semantics of probabilistic programs , 1979, 20th Annual Symposium on Foundations of Computer Science (sfcs 1979).

[49]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[50]  John H Reif.  Optimal Parallel Algorithms for Interger Sorting and Graph Connectivity. , 1985 .

[51]  Torben Hagerup,et al.  Towards Optimal Parallel Bucket Sorting , 1987, Inf. Comput..

[52]  H. T. Kung,et al.  Sorting on a mesh-connected parallel computer , 1976, STOC '76.

[53]  C. Q. Lee,et al.  The Computer Journal , 1958, Nature.

[54]  John H. Reif,et al.  Depth-First Search is Inherently Sequential , 1985, Inf. Process. Lett..

[55]  Leonard M. Adleman,et al.  Reducibility, randomness, and intractibility (Abstract) , 1977, STOC '77.

[56]  Michael Luby,et al.  A simple parallel algorithm for the maximal independent set problem , 1985, STOC '85.

[57]  Sanguthevar Rajasekaran,et al.  An Optimal Randomized Routing Algorithm for the Mesh and A Class of Efficient Mesh-like Routing Networks , 1987, FSTTCS.

[58]  J. Reif,et al.  Parallel Tree Contraction Part 1: Fundamentals , 1989, Adv. Comput. Res..

[59]  Abhiram G. Ranade,et al.  How to emulate shared memory , 1991, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[60]  Donald E. Knuth,et al.  The art of computer programming. Vol.2: Seminumerical algorithms , 1981 .

[61]  William L. Scherlis,et al.  Expression procedures and program derivation , 1980 .

[62]  Uzi Vishkin,et al.  Randomized speed-ups in parallel computation , 2015, STOC '84.

[63]  John H. Reif,et al.  An optimal parallel algorithm for integer sorting , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[64]  Narsingh Deo,et al.  Graph Theory with Applications to Engineering and Computer Science , 1975, Networks.

[65]  Alok Aggarwal,et al.  A random NC algorithm for depth first search , 1987, Comb..

[66]  Gary L. Miller,et al.  Parallel tree contraction and its application , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[67]  Frank Harary,et al.  Graph Theory , 2016 .

[68]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[69]  Franco P. Preparata,et al.  New Parallel-Sorting Schemes , 1978, IEEE Transactions on Computers.

[70]  W. Donald Frazer,et al.  Samplesort: A Sampling Approach to Minimal Storage Tree Sorting , 1970, JACM.

[71]  John H. Reif.  On Synchronous Parallel Computations with Independent Probabilistic Choice , 1984, SIAM J. Comput..

[72]  Sanguthevar Rajasekaran,et al.  Optimal and Sublogarithmic Time Randomized Parallel Sorting Algorithms , 1989, SIAM J. Comput..

[73]  Hillel Gazit,et al.  An optimal randomized parallel algorithm for finding connected components in a graph , 1986, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[74]  N.R. Malik,et al.  Graph theory with applications to engineering and computer science , 1975, Proceedings of the IEEE.

[75]  Eli Upfal,et al.  The complexity of parallel computation on matroids , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[76]  Frank Thomson Leighton,et al.  Tight Bounds on the Complexity of Parallel Sorting , 1984, IEEE Transactions on Computers.

[77]  Robert E. Tarjan,et al.  Applications of a planar separator theorem , 1977, 18th Annual Symposium on Foundations of Computer Science (sfcs 1977).

[78]  W. T. Tutte.  The Factorization of Linear Graphs , 1947 .

[79]  Eli Upfal,et al.  Parallel hashing: an efficient implementation of shared memory , 1988, JACM.

[80]  Noga Alon,et al.  The average complexity of deterministic and randomized parallel comparison sorting algorithms , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[81]  Kurt Mehlhorn,et al.  Deterministic Simulation of Idealized Parallel Computers on More Realistic Ones , 1987, SIAM J. Comput..

[82]  Sanguthevar Rajasekaran,et al.  Optimal routing algorithms for mesh-connected processor arrays , 1988, Algorithmica.

[83]  Ludek Kucera,et al.  Parallel Computation and Conflicts in Memory Access , 1982, Information Processing Letters.

[84]  David G. Kirkpatrick,et al.  Upper Bounds for Sorting Integers on Random Access Machines , 1984, Theor. Comput. Sci..

[85]  Uzi Vishkin,et al.  An O(log n) Parallel Connectivity Algorithm , 1982, J. Algorithms.