Data Redistribution Algorithms for Heterogeneous Processor Rings

We consider the problem of redistributing data on homogeneous and heterogeneous rings of processors. The problem arises in several applications, after each invocation of a load-balancing mechanism (but we do not discuss the load-balancing mechanism itself). We provide algorithms that aim at optimizing the data redistribution, both for unidirectional and bidirectional rings. One major contribution of the paper is that we are able to prove the optimality of the proposed algorithms in all cases except that of a bidirectional heterogeneous ring, for which the problem remains open.

[1]  Min-You Wu,et al.  On Runtime Parallel Scheduling for Processor Load Balancing , 1997, IEEE Trans. Parallel Distributed Syst..

[2]  Y. F. Hu,et al.  Load Balancing for Unstructured Mesh Applications , 1999, Scalable Comput. Pract. Exp..

[3]  Yves Robert,et al.  Scheduling Block-Cyclic Array Redistribution , 1998, IEEE Trans. Parallel Distributed Syst..

[4]  Yves Robert,et al.  A Proposal for a Heterogeneous Cluster ScaLAPACK (Dense Linear Solvers) , 2001, IEEE Trans. Computers.

[5]  Ulrich Kremer,et al.  NP-completeness of Dynamic Remapping , 1993 .

[6]  Serge Miguet,et al.  ARAMIS: A Remote Access Medical Imaging System , 1999, ISCOPE.

[7]  Yves Robert,et al.  Data Redistribution Algorithms for Heterogeneous Processor Rings , 2006, Int. J. High Perform. Comput. Appl..

[8]  Alessandro Bevilacqua.  A Dynamic Load Balancing Method On A Heterogeneous Cluster Of Workstations , 1999, Informatica.

[9]  Rajeev Thakur,et al.  Efficient Algorithms for Array Redistribution , 1996, IEEE Trans. Parallel Distributed Syst..

[10]  Ali R. Hurson,et al.  Scheduling and Load Balancing in Parallel and Distributed Systems , 1995 .

[11]  Zhiling Lan,et al.  Dynamic Load Balancing of SAMR Applications on Distributed Systems , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[12]  Guy L. Steele,et al.  The High Performance Fortran Handbook , 1993 .

[13]  Jack Dongarra,et al.  ScaLAPACK Users' Guide , 1987 .

[14]  Jorge G. Barbosa,et al.  Linear algebra algorithms in a heterogeneous cluster of personal computers , 2000, Proceedings 9th Heterogeneous Computing Workshop (HCW 2000) (Cat. No.PR00556).

[15]  Ching-Hsien Hsu,et al.  A Generalized Processor Mapping Technique for Array Redistribution , 2001, IEEE Trans. Parallel Distributed Syst..

[16]  J. D. Teresco,et al.  Parallel structures and dynamic load balancing for adaptive finite element computation , 1998 .

[17]  Matthew Doar,et al.  A better model for generating test networks , 1996, Proceedings of GLOBECOM'96. 1996 IEEE Global Telecommunications Conference.

[18]  Jens Knoop,et al.  Distribution Assignment Placement: Effective Optimization of Redistribution Costs , 2002, IEEE Trans. Parallel Distributed Syst..

[19]  Mounir Hamdi,et al.  Dynamic load balancing of data parallel applications on a distributed network , 1995, ICS '95.

[20]  Eduard Ayguadé,et al.  A Framework for Integrating Data Alignment, Distribution, and Redistribution in Distributed Memory Multiprocessors , 2001, IEEE Trans. Parallel Distributed Syst..

[21]  Joel H. Saltz,et al.  Dynamic Remapping of Parallel Computations with Varying Resource Demands , 1988, IEEE Trans. Computers.

[22]  Henri Casanova,et al.  Scheduling distributed applications: the SimGrid simulation framework , 2003, CCGrid 2003. 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, 2003. Proceedings..

[23]  Lionel M. Ni,et al.  Processor Mapping Techniques Toward Efficient Data Redistribution , 1995, IEEE Trans. Parallel Distributed Syst..

[24]  Vipin Kumar,et al.  A Unified Algorithm for Load-balancing Adaptive Scientific Simulations , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[25]  Mounir Hamdi,et al.  Parallel Image Processing Applications on a Network of Workstations , 1995, Parallel Comput..

[26]  Stephen Taylor,et al.  A Practical Approach to Dynamic Load Balancing , 1998, IEEE Trans. Parallel Distributed Syst..

[27]  Boniface Nkonga,et al.  Dynamic Load Balancing Computation of Pulses Propagating in a Nonlinear Medium , 2002, Proceedings. International Conference on Parallel Processing Workshop.

[28]  Harald Kosch Andr.  New static scheduling and elastic load balancing methods for parallelquery processingLionel , 1995 .

[29]  E. Deelman,et al.  Dynamic load balancing in parallel discrete event simulation for spatially explicit problems , 1998, Proceedings. Twelfth Workshop on Parallel and Distributed Simulation PADS '98 (Cat. No.98TB100233).

[30]  Sanjay Ranka,et al.  Array Decompositions for Nonuniform Computational Environments , 1996, J. Parallel Distributed Comput..

[31]  Boleslaw K. Szymanski,et al.  Adaptive Local Refinement with Octree Load Balancing for the Parallel Solution of Three-Dimensional Conservation Laws , 1997, J. Parallel Distributed Comput..

[32]  Yves Robert,et al.  Elastic Load-Balancing for Image Processing Algorithms , 1991, ACPC.

[33]  Rajkumar Buyya,et al.  High Performance Cluster Computing , 1999 .

[34]  Bernard Tourancheau,et al.  Fast Runtime Block Cyclic Data Redistribution on Multiprocessors , 1997, J. Parallel Distributed Comput..

[35]  Kenneth L. Calvert,et al.  Modeling Internet topology , 1997, IEEE Commun. Mag..

[36]  Vipin Kumar,et al.  Multilevel Diffusion Schemes for Repartitioning of Adaptive Meshes , 1997, J. Parallel Distributed Comput..

[37]  Michael J. Quinn,et al.  Block data decomposition for data-parallel programming on a heterogeneous workstation network , 1993, [1993] Proceedings The 2nd International Symposium on High Performance Distributed Computing.

[38]  David M. Nicol,et al.  Optimal Dynamic Remapping of Data Parallel Computations , 1990, IEEE Trans. Computers.

[39]  Viktor K. Prasanna,et al.  Efficient Algorithms for Block-Cyclic Array Redistribution between Processor Sets , 1998, Proceedings of the IEEE/ACM SC98 Conference.

[40]  Yves Robert,et al.  Matrix Multiplication on Heterogeneous Platforms , 2001, IEEE Trans. Parallel Distributed Syst..