Dynamic Load Balancing on Dedicated Heterogeneous Systems

Parallel computing in heterogeneous environments is drawing considerable attention due to the growing number of these kind of systems. Adapting existing code and libraries to such systems is a fundamental problem. The performance of this code is affected by the large interdependence between the code and these parallel architectures. We have developed a dynamic load balancing library that allows parallel code to be adapted to heterogeneous systems for a wide variety of problems. The overhead introduced by our system is minimal and the cost to the programmer negligible. The strategy was validated on several problems to confirm the soundness of our proposal.

[1]  Victor Eijkhout,et al.  Self-adapting numerical software (SANS) effort , 2006, IBM J. Res. Dev..

[2]  Enrique Alba,et al.  Efficient parallel LAN/WAN algorithms for optimization. The mallba project , 2006, Parallel Comput..

[3]  Xingfu Wu,et al.  Performance Evaluation, Prediction and Visualization of Parallel Systems , 1999, The Kluwer International Series on Asian Studies in Computer and Information Science.

[4]  Francisco Almeida,et al.  The master-slave paradigm on heterogeneous systems: A dynamic programming approach for the optimal mapping , 2006, J. Syst. Archit..

[5]  Alexey L. Lastovetsky,et al.  Adaptive parallel computing on heterogeneous networks with mpC , 2002, Parallel Comput..

[6]  Alexey L. Lastovetsky,et al.  HeteroMPI: Towards a message-passing library for heterogeneous networks of computers , 2006, J. Parallel Distributed Comput..

[7]  Yves Robert,et al.  Heterogeneous computing , 2005, Parallel Comput..

[8]  Rafael Mayo,et al.  Parallelization of the GNU Scientific Library on heterogeneous systems , 2004, Third International Symposium on Parallel and Distributed Computing/Third International Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogeneous Networks.

[9]  Alexey Kalinov.  Scalability of heterogeneous parallel systems , 2006, Programming and Computer Software.

[10]  Luis Pastor,et al.  Dynamic load balancing in heterogeneous clusters , 2004, Parallel and Distributed Computing and Networks.

[11]  Michael Allen,et al.  Parallel programming: techniques and applications using networked workstations and parallel computers , 1998 .

[12]  M. Nakazawa,et al.  Dyn-MPI: Supporting MPI on Non Dedicated Clusters , 2003, ACM/IEEE SC 2003 Conference (SC'03).

[13]  Jameela Al-Jaroodi,et al.  Modeling parallel applications performance on heterogeneous systems , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[14]  Javier Cuenca,et al.  Heuristics for work distribution of a homogeneous parallel dynamic programming scheme on heterogeneous systems , 2004, Third International Symposium on Parallel and Distributed Computing/Third International Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogeneous Networks.