Systematic Efficient Parallelization of Scan and Other List Homomorphisms

Homomorphisms are functions which can be parallelized by the divide-and-conquer paradigm. A class of distributable homomorphisms (DH) is introduced and an efficient parallel implementation schema for all functions of the class is derived by transformations in the Bird-Meertens formalism. The schema can be directly mapped on the hypercube with an unlimited or an arbitrary fixed number of processors, providing provable correctness and predictable performance. The popular scan-function (parallel prefix) illustrates the presentation: the systematically derived implementation for scon coincides with the practically used “folklore” algorithm for distributed-memory machines.

[1]  Franco P. Preparata,et al.  The cube-connected-cycles: A versatile network for parallel computation , 1979, 20th Annual Symposium on Foundations of Computer Science (sfcs 1979).

[2]  Guy E. Blelloch,et al.  Scans as Primitive Parallel Operations , 1989, ICPP.

[3]  Richard S. Bird,et al.  Lectures on Constructive Functional Programming , 1989 .

[4]  Z. G. Mou,et al.  Divacon: a parallel language for scientific computing based on divide-and-conquer , 1990, [1990 Proceedings] The Third Symposium on the Frontiers of Massively Parallel Computation.

[5]  Jeremy Gibbons.  Upwards and Downwards Accumulations on Trees , 1992, MPC.

[6]  Margaret Reid-Miller,et al.  List ranking and list scan on the Cray C-90 , 1994, SPAA '94.

[7]  Jayadev Misra,et al.  Powerlist: a structure for parallel recursion , 1994, TOPL.

[8]  David W. Walker,et al.  The Design of a Standard Message Passing Interface for Distributed Memory Concurrent Computers , 1994, Parallel Comput..

[9]  John T. O'Donnell,et al.  A Correctness Proof of Parallel Scan , 1994, Parallel Process. Lett..

[10]  Jacob Kornerup.  Mapping a Functional Notation for Parallel Programs Onto Hypercubes , 1995, Inf. Process. Lett..

[11]  Wentong Cai,et al.  A Cost Calculus for Parallel Functional Programming , 1995, J. Parallel Distributed Comput..

[12]  Wolfram Schulte,et al.  Architecture Independent Massive Parallelization of Divide-and-Conquer Algorithms , 1995, MPC.

[13]  S. Gorlatch.  Stages and transformations in parallel programming , 1997 .