A survey of PRAM simulation techniques

The Parallel Random Access Machine (PRAM) is an abstract model of parallel computation which allows researchers to focus on the essential characteristics of a parallel architecture and ignore other details. The PRAM has long been acknowledged to be a useful tool for the study of parallel computing, but unfortunately it is not physically implementable in hardware. In order to take advantage of the broad base of algorithms and results regarding this high-level abstraction one needs general methods for allowing the execution of PRAM algorithms on more realistic machines. In the following we survey these methods, which we refer to as PRAM simulation techniques. The general issues of memory management and routing are discussed, and both randomized and deterministic solutions are considered. We show that good theoretical solutions to many of the subproblems in PRAM simulation have been developed, though questions still exist as to their practical utility. This article should allow those performing research in this field to become well acquainted with the current state of the art, while allowing the novice to get an intuitive feeling for the fundamental questions being considered. The introduction should provide a concise tutorial for those unfamiliar with the problem of PRAM simulation.

[1]  Nicholas Pippenger,et al.  Parallel Communication with Limited Buffers (Preliminary Version) , 1984, FOCS.

[2]  A Probabilistic Simulation of PRAMs on a Bounded Degree Network , 1988, Inf. Process. Lett..

[3]  Stephen A. Cook,et al.  A Taxonomy of Problems with Fast Parallel Algorithms , 1985, Inf. Control..

[4]  Kenneth E. Batcher,et al.  Sorting networks and their applications , 1968, AFIPS Spring Joint Computing Conference.

[5]  Bruce M. Maggs,et al.  Expanders might be practical: fast algorithms for routing around faults on multibutterflies , 1989, 30th Annual Symposium on Foundations of Computer Science.

[6]  Sandeep N. Bhatt,et al.  The fluent abstract machine , 1988 .

[7]  Danny Krizanc.  Oblivious Routing with Limited Buffer Capacity , 1991, J. Comput. Syst. Sci..

[8]  Michael O. Rabin,et al.  Efficient dispersal of information for security, load balancing, and fault tolerance , 1989, JACM.

[9]  Allan Borodin,et al.  Routing, Merging, and Sorting on Parallel Models of Computation , 1985, J. Comput. Syst. Sci..

[10]  Uzi Vishkin,et al.  Finding the Maximum, Merging, and Sorting in a Parallel Computation Model , 1981, J. Algorithms.

[11]  Bruce M. Maggs,et al.  On-line algorithms for path selection in a nonblocking network , 1990, STOC '90.

[12]  Alok Aggarwal,et al.  Communication Complexity of PRAMs , 1990, Theor. Comput. Sci..

[13]  Richard Cole,et al.  Parallel merge sort , 1988, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[14]  Uzi Vishkin,et al.  Implementation of Simultaneous Memory Address Access in Models That Forbid It , 1983, J. Algorithms.

[15]  Richard M. Karp,et al.  Parallel Algorithms for Shared-Memory Machines , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[16]  Abhiram G. Ranade,et al.  How to emulate shared memory , 1991, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[17]  Friedhelm Meyer auf der Heide,et al.  How to distribute a dictionary in a complete network , 1990, STOC '90.

[18]  Gianfranco Bilardi,et al.  Deterministic Simulations of PRAMs on Bounded Degree Networks , 1994, SIAM J. Comput..

[19]  Leslie G. Valiant,et al.  Universal schemes for parallel communication , 1981, STOC '81.

[20]  Fabrizio Luccio,et al.  Analysis of Parallel Uniform Hashing , 1991, Inf. Process. Lett..

[21]  Debasis Mitra,et al.  Randomized Parallel Communications , 1986, ICPP.

[22]  Yonatan Aumann,et al.  Improved Memory Utilization in Deterministic PRAM Simulation , 1991, J. Parallel Distributed Comput..

[23]  Larry Rudolph,et al.  A Complexity Theory of Efficient Parallel Algorithms , 1990, Theor. Comput. Sci..

[24]  Stephen A. Cook,et al.  The Classifikation of Problems which have Fast Parallel Algorithms , 1983, FCT.

[25]  Kurt Mehlhorn,et al.  Deterministic Simulation of Idealized Parallel Computers on More Realistic Ones , 1986, SIAM J. Comput..

[26]  Friedhelm Meyer auf der Heide,et al.  Efficient PRAM simulation on a distributed memory machine , 1992, STOC '92.

[27]  Eli Upfal,et al.  A probabilistic relation between desirable and feasible, models of parallel computation , 1984, STOC '84.

[28]  Robert H. Thomas,et al.  A Majority consensus approach to concurrency control for multiple copy databases , 1979, ACM Trans. Database Syst..

[29]  Leslie M. Goldschlager,et al.  A unified approach to models of synchronous parallel machines , 1978, STOC.

[30]  Kurt Mehlhorn,et al.  Deterministic Simulation of Idealized Parallel Computers on More Realistic Ones , 1986, MFCS.

[31]  János Komlós,et al.  Sorting in c log n parallel sets , 1983, Comb..

[32]  W. F. McColl,et al.  General purpose parallel computing , 1993 .

[33]  Wojciech Rytter,et al.  Efficient parallel algorithms , 1988 .

[34]  Leslie G. Valiant,et al.  A Scheme for Fast Parallel Communication , 1982, SIAM J. Comput..

[35]  Larry Carter,et al.  Universal Classes of Hash Functions , 1979, J. Comput. Syst. Sci..

[36]  Frank Thomson Leighton,et al.  Tight Bounds on the Complexity of Parallel Sorting , 1985, IEEE Trans. Computers.

[37]  Michael T. Goodrich,et al.  A bridging model for parallel computation, communication, and I/O , 1996, CSUR.

[38]  Steven Fortune,et al.  Parallelism in random access machines , 1978, STOC.

[39]  Uzi Vishkin,et al.  Converting high probability into nearly-constant time—with applications to parallel hashing , 1991, STOC '91.

[40]  Kieran T. Herley,et al.  Efficient simulations of small shared memories on bounded degree networks , 1989, 30th Annual Symposium on Foundations of Computer Science.

[41]  Franco P. Preparata,et al.  Deterministic P-RAM Simulation with Constant Redundancy , 1991, Inf. Comput..

[42]  A. Gupta,et al.  Exploring the benefits of multiple hardware contexts in a multiprocessor architecture: preliminary results , 1989, ISCA '89.

[43]  Leslie G. Valiant,et al.  A bridging model for parallel computation , 1990, CACM.

[44]  Uzi Vishkin,et al.  Finding the maximum, merging and sorting in a parallel computation model , 1981, CONPAR.

[45]  Eli Upfal,et al.  An O(log N) deterministic packet-routing scheme , 1992, JACM.

[46]  Noam Nisan,et al.  The computational complexity of universal hashing , 1990, Proceedings Fifth Annual Structure in Complexity Theory Conference.

[47]  Leslie M. Goldschlager,et al.  A universal interconnection pattern for parallel computers , 1982, JACM.

[48]  Eli Upfal,et al.  Efficient schemes for parallel communication , 1982, PODC '82.

[49]  Leslie G. Valiant,et al.  Optimality of a Two-Phase Strategy for Routing in Interconnection Networks , 1983, IEEE Transactions on Computers.

[50]  J. L. C. Sanz.  Opportunities and Constraints of Parallel Computing , 1989, Springer US.

[51]  Alan Siegel,et al.  On universal classes of fast high performance hash functions, their time-space tradeoff, and their applications , 1989, 30th Annual Symposium on Foundations of Computer Science.

[52]  Uzi Vishkin,et al.  A Parallel-Design Distributed-Implementation (PDDI) General-Purpose Computer , 2011, Theor. Comput. Sci..

[53]  Eli Upfal,et al.  How to share memory in a distributed system , 1984, JACM.

[54]  Kurt Mehlhorn,et al.  Deterministic Simulation of Idealized Parallel Computers on More Realistic Ones , 1987, SIAM J. Comput..

[55]  Ludek Kucera,et al.  Parallel Computation and Conflicts in Memory Access , 1982, Information Processing Letters.

[56]  Romas Aleliunas,et al.  Randomized parallel communication (Preliminary Version) , 1982, PODC '82.

[57]  Eli Upfal,et al.  Parallel hashing: an efficient implementation of shared memory , 1988, JACM.

[58]  Gregory F. Pfister,et al.  “Hot spot” contention and combining in multistage interconnection networks , 1985, IEEE Transactions on Computers.

[59]  Noga Alon,et al.  Eigenvalues and expanders , 1986, Comb..

[60]  Leslie G. Valiant,et al.  General Purpose Parallel Architectures , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.