Algorithm-system scalability of heterogeneous computing

Scalability is a key factor of the design of distributed systems and parallel algorithms and machines. However, conventional scalabilities are designed for homogeneous parallel processing. There is no suitable and commonly accepted definition of scalability metric for heterogeneous systems. Isospeed scalability is a well-defined metric for homogeneous computing. This study extends the isospeed scalability metric to general heterogeneous computing systems. The proposed isospeed-efficiency model is suitable for both homogeneous and heterogeneous computing. Through theoretical analyses, we derive methodologies of scalability measurement and prediction for heterogeneous systems. Experimental results have verified the analytical results and confirmed that the proposed isospeed-efficiency scalability works well in both homogeneous and heterogeneous environments.

[1]  Arie Shoshani,et al.  The Grid 2: Blueprint for a New Computing Infrastructure (2nd edition), , 2003 .

[2]  Vipin Kumar,et al.  Scalability of Parallel Algorithms for Matrix Multiplication , 1993, 1993 International Conference on Parallel Processing - ICPP'93.

[3]  Yong Yan,et al.  Latency Metric: An Experimental Method for Measuring and Evaluating Parallel Program and Architecture Scalability , 1994, J. Parallel Distributed Comput..

[4]  Hugh Garraway.  Parallel Computer Architecture: A Hardware/Software Approach , 1999, IEEE Concurrency.

[5]  Alexey L. Lastovetsky,et al.  Heterogeneous Distribution of Computations While Solving Linear Algebra Problems on Networks of Heterogeneous Computers , 1999, HPCN Europe.

[6]  Xian-He Sun,et al.  Scalability of Parallel Algorithm-Machine Combinations , 1994, IEEE Trans. Parallel Distributed Syst..

[7]  José Luis Bosque,et al.  Theoretical scalability analysis for heterogeneous clusters , 2004, IEEE International Symposium on Cluster Computing and the Grid, 2004. CCGrid 2004..

[8]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[9]  George Karypis,et al.  Introduction to Parallel Computing , 1994 .

[10]  Keqin Li,et al.  Average-Case Scalability Analysis of Parallel Computations , 2002 .

[11]  Alexey L. Lastovetsky,et al.  Heterogeneous Distribution of Computations Solving Linear Algebra Problems on Networks of Heterogeneous Computers , 2001, J. Parallel Distributed Comput..

[12]  Jianping Zhu,et al.  Parallel architectures: Performance prediction: A case study using a scalable shared-virtual-memory machine , 1996, IEEE Parallel & Distributed Technology: Systems & Applications.

[13]  Ming Wu,et al.  Scalability of heterogeneous computing , 2005, 2005 International Conference on Parallel Processing (ICPP'05).

[14]  Xian-He Sun.  Scalability versus Execution Time in Scalable Systems , 2002, J. Parallel Distributed Comput..

[15]  Yves Robert,et al.  Matrix Multiplication on Heterogeneous Platforms , 2001, IEEE Trans. Parallel Distributed Syst..

[16]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[17]  Luis Pastor,et al.  An efficiency and scalability model for heterogeneous clusters , 2001, Proceedings 42nd IEEE Symposium on Foundations of Computer Science.

[18]  C. Murray Woodside,et al.  Evaluating the Scalability of Distributed Systems , 2000, IEEE Trans. Parallel Distributed Syst..

[19]  Xian-He Sun,et al.  STAS: A Scalability Testing and Analysis System , 2006, 2006 IEEE International Conference on Cluster Computing.

[20]  Yong Yan,et al.  Modeling and characterizing parallel computing performance on heterogeneous networks of workstations , 1995, Proceedings.Seventh IEEE Symposium on Parallel and Distributed Processing.

[21]  Pankaj Mehra,et al.  Automated scalability analysis of message-passing parallel programs , 1995, IEEE Parallel Distributed Technol. Syst. Appl..

[22]  Lionel M. Ni,et al.  Scalable Problems and Memory-Bounded Speedup , 1993, J. Parallel Distributed Comput..