Supporting tasks with adaptive groups in data parallel programming

A set of communication operations is defined, which allows a form of task parallelism to be achieved in a data parallel architecture. The set of processors can be subdivided recursively into groups, and a communication operation inside a group never conflicts with communications taking place in other groups. The groups may be subdivided and recombined at any time, allowing the task structure to adapt to the needs of the data. The algorithms implementing the grouping and communications are defined using parallel scans and folds which can be executed efficiently in an abstract tree machine. This approach is best suited for massively parallel systems with fine grain processors.

[1]  Guy E. Blelloch,et al.  Scans as Primitive Parallel Operations , 1989, ICPP.

[2]  John T. O'Donnell,et al.  A Correctness Proof of Parallel Scan , 1994, Parallel Process. Lett..

[3]  A. L. Decegama.  The Technology of Parallel Processing , 1989 .

[4]  Allan Gottlieb,et al.  Highly parallel computing , 1989, Benjamin/Cummings Series in computer science and engineering.

[5]  Larry Rudolph,et al.  The power of parallel prefix , 1985, IEEE Transactions on Computers.

[6]  Thomas Rauber,et al.  A Transformation Approach to Derive Efficient Parallel Implementations , 2000, IEEE Trans. Software Eng..

[7]  Jonathan Rose,et al.  Mixing buffers and pass transistors in FPGA routing architectures , 2001, FPGA '01.

[8]  Jerry L. Potter.  Associative Computing: A Programming Paradigm for Massively Parallel Computers , 1992 .

[9]  Matthew Haines,et al.  Approaches for integrating task and data parallelism , 1998, IEEE Concurr..

[10]  Gyula A. Magó.  Copying operands versus copying results: A solution to the problem of large operands in FFP'S , 1981, FPCA '81.

[11]  Kai Hwang,et al.  Computer architecture and parallel processing , 1984, McGraw-Hill Series in computer organization and architecture.

[12]  Gregory V. Wilson,et al.  Practical parallel programming , 1999, IEEE Concurr..

[13]  John T. O'Donnell.  A systolic associative lisp computer architecture with incremental parallel storage management , 1981 .

[14]  Peyton Jones,et al.  Haskell 98 language and libraries : the revised report , 2003 .

[15]  Guy E. Blelloch,et al.  Programming parallel algorithms , 1996, CACM.

[16]  David B. Skillicorn,et al.  Models and languages for parallel computation , 1998, CSUR.

[17]  John E. Savage,et al.  Models of computation - exploring the power of computing , 1998 .

[18]  S. Lakshmivarahan,et al.  Parallel Sorting Algorithms , 1984, Adv. Comput..

[19]  W. Daniel Hillis,et al.  The network architecture of the Connection Machine CM-5 (extended abstract) , 1992, SPAA '92.

[20]  W. Daniel Hillis,et al.  The Network Architecture of the Connection Machine CM-5 , 1996, J. Parallel Distributed Comput..

[21]  Scott Hauck,et al.  Reconfigurable computing: a survey of systems and software , 2002, CSUR.

[22]  Salvatore J. Stolfo,et al.  A parallel and distributed environment for database rule processing: open problems and future directions , 1995 .

[23]  Selim G. Akl,et al.  Design and analysis of parallel algorithms , 1985 .

[24]  Gyula A. Magó.  Data sharing in an FFP machine , 1982, LFP '82.

[25]  Joyce L. Vedral,et al.  Functional Programming Languages and Computer Architecture , 1989, Lecture Notes in Computer Science.

[26]  David A. Patterson,et al.  X-Tree: A tree structured multi-processor computer architecture , 1978, ISCA '78.

[27]  Guy E. Blelloch,et al.  Vector Models for Data-Parallel Computing , 1990 .

[28]  Jerry L. Potter.  The Massively Parallel Processor , 1985 .

[29]  John T. O'Donnell,et al.  A VLSI implementation of an architecture for applicative programming , 1988, Future Gener. Comput. Syst..

[30]  Philip J. Hatcher,et al.  Data-Parallel Programming on MIMD Computers , 1991, IEEE Trans. Parallel Distributed Syst..

[31]  W. Daniel Hillis,et al.  The connection machine , 1985 .