Class Notes : Programming Parallel Algorithms

These are the lecture notes for CS 15-840B, a hands-on class in programming parallel algorithms. The class was taught in the fall of 1992 by Guy Blelloch, using the programming language NESL. It stressed the clean and concise expression of a variety of parallel algorithms. About 35 graduate students attended the class, of whom 28 took it for credit. These notes were written by students in the class, and were then reviewed and organized by Guy Blelloch and Jonathan Hardwick. The sample NESL code has been converted from the older LISP-style syntax into the new ML-style syntax. These notes are not in a polished form, and probably contain several errors and omissions, particularly with respect to references in the literature. Corrections are welcome. This research was sponsored in part by the Avionics Laboratory, Wright Research and Development Center, Aeronautical Systems Division (AFSC), U.S. Air Force, Wright-Patterson AFB, Ohio 45433-6543 under Contract F33615-90-C-1465,ARPA Order No. 7597 and in part by the Pittsburgh Supercomputing Center (Grant ASC890018P), who provided Connection Machine CM-2 and Cray Y-MP time. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. government.

[1]  Richard P. Brent,et al.  The Parallel Evaluation of General Arithmetic Expressions , 1974, JACM.

[2]  H. T. Kung.  Systolic algorithms for the CMU warp processor , 1984 .

[3]  L. W. Tucker,et al.  Object recognition using the Connection Machine , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  S. Sitharama Iyengar,et al.  Introduction to parallel algorithms , 1998, Wiley series on parallel and distributed computing.

[6]  Leonidas J. Guibas,et al.  Parallel computational geometry , 1988, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[7]  A. W. M. van den Enden,et al.  Discrete Time Signal Processing , 1989 .

[8]  Guy E. Blelloch,et al.  Prefix sums and their applications , 1990 .

[9]  Kenneth E. Batcher,et al.  Sorting networks and their applications , 1968, AFIPS Spring Joint Computing Conference.

[10]  Uzi Vishkin,et al.  Deterministic sampling—a new technique for fast pattern matching , 1990, STOC '90.

[11]  Zvi Galil,et al.  An Optimal O(log log n) Time Parallel String Matching Algorithm , 1990, SIAM J. Comput..

[12]  Guy E. Blelloch,et al.  NESL: A Nested Data-Parallel Language , 1992 .

[13]  W. Daniel Hillis,et al.  Data parallel algorithms , 1986, CACM.

[14]  Abhiram G. Ranade,et al.  How to emulate shared memory (Preliminary Version) , 1987, FOCS.

[15]  James J. Little,et al.  Algorithmic Techniques for Computer Vision on a Fine-Grained Parallel Machine , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  F. Leighton,et al.  Introduction to Parallel Algorithms and Architectures: Arrays, Trees, Hypercubes , 1991 .

[17]  John H Reif.  Optimal Parallel Algorithms for Interger Sorting and Graph Connectivity. , 1985 .

[18]  Jan van Leeuwen,et al.  Maintenance of Configurations in the Plane , 1981, J. Comput. Syst. Sci..

[19]  Bent E. Petersen.  Introduction to the Fourier transform & pseudo-differential operators , 1983 .

[20]  Mihalis Yannakakis,et al.  High-probability parallel transitive closure algorithms , 1990, SPAA '90.

[21]  Mikhail J. Atallah,et al.  Efficient Parallel Solutions to Geometric Problems , 1985, ICPP.

[22]  Joseph JáJá,et al.  Fast, Efficient Parallel Algorithms for Some Graph Problems , 1981, SIAM J. Comput..

[23]  Michael L. Baird.  Structural Pattern Recognition , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Joseph JáJá,et al.  An Introduction to Parallel Algorithms , 1992 .

[25]  Arthur L. Delcher,et al.  Optimal Parallel Evaluation of Tree-Structured Computations by Raking , 1988, AWOC.

[26]  Guy E. Blelloch,et al.  A comparison of sorting algorithms for the connection machine CM-2 , 1991, SPAA '91.

[27]  Gary L. Miller,et al.  Parallel tree contraction and its application , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[28]  Ralph Grishman,et al.  The NYU ultracomputer—designing a MIMD, shared-memory parallel machine , 2018, ISCA '98.

[29]  Guy E. Blelloch,et al.  Vector Models for Data-Parallel Computing , 1990 .

[30]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[31]  Guy E. Blelloch,et al.  Programming parallel algorithms , 1996, CACM.

[32]  Abhiram G. Ranade,et al.  How to emulate shared memory , 1991, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[33]  Michael Ian Shamos,et al.  Computational geometry: an introduction , 1985 .

[34]  Larry J. Stockmeyer,et al.  A characterization of the power of vector machines , 1974, STOC '74.

[35]  Robert E. Tarjan,et al.  An Efficient Parallel Biconnectivity Algorithm , 2011, SIAM J. Comput..