Performance models for scalable cluster computing

Abstract In this paper, we make a thorough study of scalable parallel computing performance models, point out some of the weaknesses of existing performance models, and discuss how one can improve and generalize them. We start with analyzing and accurately defining several kinds of serial and parallel execution times, discuss parallel speedup models, and present a generalized speedup model. We propose a formal definition of scalability, and discuss scalability of cluster systems from three aspects. As an extension to the isoefficiency metric and isospeed metric, we propose and define scalability of parallel algorithm-architecture combinations based on the fixed ratio of computation time to communication overhead, analyze some relevant theoretical perspectives, and compare scalability with speedup.

[1]  Frederic A. Van-Catledge.  Toward a General Model for Evaluating the Relative Performance of Computer Systems , 1989, Int. J. High Perform. Comput. Appl..

[2]  Lionel M. Ni,et al.  Scalable Problems and Memory-Bounded Speedup , 1993, J. Parallel Distributed Comput..

[3]  Xian-He Sun,et al.  Scalability of Parallel Algorithm-Machine Combinations , 1994, IEEE Trans. Parallel Distributed Syst..

[4]  G. Amdhal,et al.  Validity of the single processor approach to achieving large scale computing capabilities , 1967, AFIPS '67 (Spring).

[5]  Gordon Bell,et al.  Ultracomputers: a teraflop before its time , 1992, CACM.

[6]  Roy M. Jenevein,et al.  Scaleability of a Binary Tree on a Hypercube , 1986, ICPP.

[7]  Steven Fortune,et al.  Parallelism in random access machines , 1978, STOC.

[8]  Yong Yan,et al.  Latency Metric: An Experimental Method for Measuring and Evaluating Parallel Program and Architecture Scalability , 1994, J. Parallel Distributed Comput..

[9]  John L. Gustafson,et al.  Reevaluating Amdahl's law , 1988, CACM.

[10]  Anand Sivasubramaniam,et al.  An approach to scalability study of shared memory parallel systems , 1994, SIGMETRICS.

[11]  Robert A. van de Geijn,et al.  Scalability Issues Affecting the Design of a Dense Linear Algebra Library , 1994, J. Parallel Distributed Comput..

[12]  Kai Hwang,et al.  Advanced computer architecture - parallelism, scalability, programmability , 1992 .

[13]  Sartaj Sahni,et al.  Performance metrics: keeping the focus on runtime , 1996, IEEE Parallel Distributed Technol. Syst. Appl..

[14]  Anoop Gupta,et al.  Scaling parallel programs for multiprocessors: methodology and examples , 1993, Computer.

[15]  G. R. Withers,et al.  Computing performance as a function of the speed, quantity, and cost of the processors , 1989, Proceedings of the 1989 ACM/IEEE Conference on Supercomputing (Supercomputing '89).

[16]  Patrick H. Worley,et al.  The Effect of Time Constraints on Scaled Speedup , 1990, SIAM J. Sci. Comput..

[17]  Dan I. Moldovan,et al.  Parallel processing - from applications to systems , 1993 .

[18]  Leslie G. Valiant,et al.  A bridging model for parallel computation , 1990, CACM.

[19]  Anant Agarwal,et al.  Scalability of parallel machines , 1991, CACM.

[20]  Xian-He Sun,et al.  Toward a better parallel performance metric , 1991, Parallel Comput..

[21]  Edward D. Lazowska,et al.  Speedup Versus Efficiency in Parallel Systems , 1989, IEEE Trans. Computers.

[22]  George Karypis,et al.  Introduction to Parallel Computing , 1994 .

[23]  Vipin Kumar,et al.  Scalable Load Balancing Techniques for Parallel Computers , 1994, J. Parallel Distributed Comput..

[24]  Vijay P. Kumar,et al.  Analyzing Scalability of Parallel Algorithms and Architectures , 1994, J. Parallel Distributed Comput..

[25]  Vipin Kumar,et al.  Isoefficiency: measuring the scalability of parallel algorithms and architectures , 1993, IEEE Parallel & Distributed Technology: Systems & Applications.

[26]  Wolfgang Rönsch,et al.  Scalability of Algorithms: An Analytic Approach , 1995, Parallel Comput..

[27]  Lionel M. Ni,et al.  Another view on parallel speedup , 1990, Proceedings SUPERCOMPUTING '90.