Natural block data decomposition for heterogeneous clusters

We propose general purposes natural heuristics for static block and block-cyclic heterogeneous data decomposition over processes of parallel program mapped into multidimensional grid. This heuristics is an extension of the intuitively clear heterogeneous data distribution for one-dimensional case. It is compared to advanced heuristics for heterogeneous data decomposition proposed for solving linear algebra problems on two-dimensional process grid. We experimentally show that for typical local network (12 Windows 2000 PCs interconnected via Fast Ethernet switch) and for typical linear algebra problems these two heuristics have almost the same efficiency. We demonstrate efficiency of the proposed natural decomposition for case of three-dimensional process grid on the example of 3D modeling of supernova explosion.

[1]  U. Ziegler.  NIRVANA+: An adaptive mesh refinement code for gas dynamics and MHD , 1998 .

[2]  Jack Dongarra,et al.  ScaLAPACK Users' Guide , 1987 .

[3]  Jorge G. Barbosa,et al.  Linear algebra algorithms in a heterogeneous cluster of personal computers , 2000, Proceedings 9th Heterogeneous Computing Workshop (HCW 2000) (Cat. No.PR00556).

[4]  Sanjay Ranka,et al.  Array Decompositions for Nonuniform Computational Environments , 1996, J. Parallel Distributed Comput..

[5]  Michael J. Quinn,et al.  Three-dimensional grid partitioning for network parallel processing , 1994, CSC '94.

[6]  Robert A. van de Geijn,et al.  SUMMA: Scalable Universal Matrix Multiplication Algorithm , 1995 .

[7]  Larry Carter,et al.  A Programming Model for Automated Decomposition on Heterogeneous Clusters of Multiprocessors , 2000 .

[8]  Jorge G. Barbosa,et al.  A group block distribution strategy for a heterogeneous machine , 2002 .

[9]  S. Ustyugov,et al.  On the neutrino mechanism of supernova explosions , 1997 .

[10]  Yves Robert,et al.  Load balancing strategies for dense linear algebra kernels on heterogeneous two-dimensional grids , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[11]  Alexey L. Lastovetsky,et al.  Heterogeneous Distribution of Computations Solving Linear Algebra Problems on Networks of Heterogeneous Computers , 2001, J. Parallel Distributed Comput..

[12]  Alexey L. Lastovetsky,et al.  A parallel language and its programming system for heterogeneous networks , 2000, Concurr. Pract. Exp..

[13]  Robert A. van de Geijn,et al.  SUMMA: scalable universal matrix multiplication algorithm , 1995, Concurr. Pract. Exp..

[14]  Michael J. Quinn,et al.  Block data decomposition for data-parallel programming on a heterogeneous workstation network , 1993, [1993] Proceedings The 2nd International Symposium on High Performance Distributed Computing.

[15]  Yves Robert,et al.  A Proposal for a Heterogeneous Cluster ScaLAPACK (Dense Linear Solvers) , 2001, IEEE Trans. Computers.