HLogGP: a new parallel computational model for heterogeneous clusters

Heterogeneous clusters claim for new models and algorithms. In this paper a new parallel computational model is presented. The model, based on the LogGP model, has been extended to be able to deal with heterogeneous parallel systems. For that purpose, the LogGP's scalar parameters have been replaced by vector and matrix parameters to take into account the different node's features. The work presented here includes the parameterization of a real cluster which illustrates the impact of node heterogeneity over the model's parameters. Finally, the paper presents some experiments performed in a real heterogeneous cluster that can be used for assessing the method's validity, together with the main conclusions and future work.

[1]  Arnold L. Rosenberg,et al.  Sharing partitionable workloads in heterogeneous NOWs: greedier is not better , 2001, Proceedings 42nd IEEE Symposium on Foundations of Computer Science.

[2]  Richard Cole,et al.  The APRAM: incorporating asynchrony into the PRAM model , 1989, SPAA '89.

[3]  Robert D. Blumofe,et al.  Scheduling large-scale parallel computations on networks of workstations , 1994, Proceedings of 3rd IEEE International Symposium on High Performance Distributed Computing.

[4]  Leslie G. Valiant,et al.  A bridging model for parallel computation , 1990, CACM.

[5]  C. R. Jesshope.  Vector Models for Data-Parallel Computing , 1991 .

[6]  David B. Skillicorn,et al.  Predictable communication on unpredictable networks: implementing BSP over TCP/IP and UDP/IP , 1999, Concurr. Pract. Exp..

[7]  Viktor K. Prasanna,et al.  Adaptive communication algorithms for distributed heterogeneous systems , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[8]  Alok Aggarwal,et al.  On communication latency in PRAM computations , 1989, SPAA '89.

[9]  Richard P. Martin,et al.  Assessing Fast Network Interfaces , 1996, IEEE Micro.

[10]  Franck Cappello,et al.  HiHCoHP-Toward a realistic communication model for hierarchical hyperclusters of heterogeneous processors , 2001, Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001.

[11]  Alok Aggarwal,et al.  Communication Complexity of PRAMs , 1990, Theor. Comput. Sci..

[12]  Paul D. Gader,et al.  Image algebra techniques for parallel image processing , 1987 .

[13]  Gordon Bell,et al.  What's next in high-performance computing? , 2002, CACM.

[14]  Amotz Bar-Noy,et al.  Designing broadcasting algorithms in the postal model for message-passing systems , 1992, SPAA '92.

[15]  John H. Reif,et al.  Models and Resource Metrics for Parallel and Distributed Computation , 1994, Parallel Algorithms Appl..

[16]  Yong Yan,et al.  Modeling and characterizing parallel computing performance on heterogeneous networks of workstations , 1995, Proceedings.Seventh IEEE Symposium on Parallel and Distributed Processing.

[17]  Arnold L. Rosenberg,et al.  An Optimal Strategies for Cycle-Stealing in Networks of Workstations , 1997, IEEE Trans. Computers.

[18]  Phillip B. Gibbons.  A more practical PRAM model , 1989, SPAA '89.

[19]  Viktor K. Prasanna,et al.  Efficient collective communication in distributed heterogeneous systems , 2003, J. Parallel Distributed Comput..

[20]  Ben H. H. Juurlink,et al.  A quantitative comparison of parallel computation models , 1996, SPAA '96.

[21]  Steven Fortune,et al.  Parallelism in random access machines , 1978, STOC.

[22]  Ramesh Subramonian,et al.  LogP: towards a realistic model of parallel computation , 1993, PPOPP '93.

[23]  Amos Fiat,et al.  Making commitments in the face of uncertainty: how to pick a winner almost every time (extended abstract) , 1996, STOC '96.

[24]  Dhabaleswar K. Panda,et al.  Efficient collective communication on heterogeneous networks of workstations , 1998, Proceedings. 1998 International Conference on Parallel Processing (Cat. No.98EX205).