Dynamic process partitioning and migration for irregular applications

Many practical applications generate irregular, nonbalanced divide-and-conquer trees which have different depths, possibly also different numbers of successors at different levels. Efficient parallelization is difficult as it requires dynamic partitioning and mapping of such trees to available processors. Irregular applications can obtain unpredictable intermediate results which then affect creation and termination of processes. The new proposed C++ framework called DAMPVM/DAC offers a combined scheme of dynamic process/data partitioning and migration which enables automatic parallelization of irregular divide-and-conquer applications taking into account processor speeds, network status, changing application requirements as well as external load introduced by other users. Experiments on a network of workstations include adaptive integration with and without process migration as well as static and dynamic codes for image recognition. The latter ones enable to assess both the overhead of the dynamic scheme compared to serial implementations for regular applications and scalability gains for non-uniform images.

[1]  Andreas Holzman.  Recent Advances in Parallel Virtual Machine and Message Passing Interface , 2001, Lecture Notes in Computer Science.

[2]  Römke Thomas,et al.  Programming frames for the efficient use of parallel systems , 1997 .

[3]  Hans-Wolfgang Loidl,et al.  On the Granularity of Divide-and-Conquer Parallelism , 1995, Functional Programming.

[4]  Jack Dongarra,et al.  Pvm 3 user's guide and reference manual , 1993 .

[5]  Paul Hudak,et al.  An algebraic model for divide-and-conquer and its parallelism , 2004, The Journal of Supercomputing.

[6]  Henryk Krawczyk,et al.  Parallel Image Matching on PC Cluster , 2001, PVM/MPI.

[7]  Kang Su Gatlin,et al.  Architecture-Cognizant Divide and Conquer Algorithms , 1999, ACM/IEEE SC 1999 Conference (SC'99).

[8]  Michael Allen,et al.  Parallel programming: techniques and applications using networked workstations and parallel computers , 1998 .

[9]  Henryk Krawczyk,et al.  Parallel program execution with process migration , 2000, Proceedings International Conference on Parallel Computing in Electrical Engineering. PARELEC 2000.

[10]  Henryk Krawczyk,et al.  Dynamic Assignment with Process Migration in Distributed Environments , 1999, PVM/MPI.

[11]  Bernd Freisleben,et al.  Automatic Parallelization of Divide-and-Conquer Algorithms , 1992, CONPAR.

[12]  Henri E. Bal,et al.  Satin: Efficient Parallel Divide-and-Conquer in Java , 2000, Euro-Par.

[13]  Rajkumar Buyya,et al.  High Performance Cluster Computing: Programming and Applications , 1999 .

[14]  Henri E. Bal,et al.  Efficient load balancing for wide-area divide-and-conquer applications , 2001, PPoPP '01.

[15]  Eric A. Brewer,et al.  ATLAS: an infrastructure for global computing , 1996, EW 7.

[16]  Shih-Hao Hung,et al.  Multiresolution time domain modeling for large scale wireless communication problems , 2001, IEEE Antennas and Propagation Society International Symposium. 2001 Digest. Held in conjunction with: USNC/URSI National Radio Science Meeting (Cat. No.01CH37229).

[17]  Stefan U. Haenssgen.  REAPAR user manual and reference: automatic parallelization of recursive programs , 1998 .

[18]  Ellis Horowitz,et al.  Divide-and-Conquer for Parallel Processing , 1983, IEEE Transactions on Computers.

[19]  Henryk Krawczyk,et al.  Dynamic Partitioning of the Divide-and-Conquer Scheme with Migration in PVM Environment , 2001, PVM/MPI.

[20]  Bradley C. Kuszmaul,et al.  Cilk: an efficient multithreaded runtime system , 1995, PPOPP '95.