Machine Characterization and Benchmark Performance Prediction

From runs of standard benchmark suites, it is not possible to characterize the machine nor to predict the running time of other benchmarks which have not been run. In this paper, we report on a new approach to benchmarking and machine characterization. We describe the creation and use of a machine analyzer, which measures the performance of a given machine on Fortran source language constructs. The machine analyzer yields a set of parameters which characterize the machine and spotlight its strong and weak points. We also describe a program analyzer, which analyzes Fortran programs and determines the frequency of execution of each of the same set of source language operations. We then show that by combining a machine characterization and a program characterization, we are able to predict with good accuracy the running time of a given benchmark on a given machine. Characterizations are provided for the Cray X-MP/48, Cyber 205, IBM 3090/200, Amdahl 5840, ConvexC-1, VAX 8600, VAX 11/785, VAX 11/ 780, SUN 3/50 and IBM RT-PC/125, and for the following benchmark programs suites: Los Alamos (BMK8A1), Baskett, Linpack, Livermore Loops, Mandelbrot Set, NAS Kernels, Shell Sort, Smith, Whetstone and Sieve of Erathostenes.

[1]  Jacob Feldman,et al.  Rejuvenating experimental computer science , 1979 .

[2]  Jack J. Dongarra.  Performance of various computers using standard linear equations software in a Fortran environment , 1983, CARN.

[3]  F. H. Mcmahon,et al.  The Livermore Fortran Kernels: A Computer Test of the Numerical Performance Range , 1986 .

[4]  Jack Dongarra,et al.  Computer benchmarking: paths and pitfalls , 1987 .

[5]  Brian A. Wichmann,et al.  A Synthetic Benchmark , 1976, Comput. J..

[6]  Michael J. Flynn,et al.  Some Computer Organizations and Their Effectiveness , 1972, IEEE Transactions on Computers.

[7]  Roland N. Ibbett.  The Architecture of High Performance Computers , 1982, Springer New York.

[8]  Chris R. Jesshope,et al.  Parallel Computers 2: Architecture, Programming and Algorithms , 1981 .

[9]  Jack J. Dongarra,et al.  Performance of various computers using standard linear equations software in a FORTRAN environment , 1988, CARN.

[10]  Myron H. MacDougall.  Instruction-Level Program and Processor Modeling , 1984, Computer.

[11]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[12]  Leigh R. Power,et al.  Design and Use of a Program Execution Analyzer , 1983, IBM Syst. J..

[13]  Reinhold Weicker,et al.  Dhrystone: a synthetic systems programming benchmark , 1984, CACM.

[14]  Alan Jay Smith,et al.  Branch Prediction Strategies and Branch Target Buffer Design , 1995, Computer.

[15]  Richard A. Volz,et al.  Toward real-time performance benchmarks for Ada , 1986, CACM.

[16]  Alan Jay Smith.  CPU Cache Memories , 1989, SIGMETRICS.

[17]  Roger W. Hockney,et al.  (r∞, n1/2, s1/2) measurements on the 2-CPU CRAY X-MP , 1985, Parallel Comput..

[18]  J. H. Griffin,et al.  Los Alamos National Laboratory computer benchmarking, 1983 , 1984 .

[19]  Michael Beeler.  Beyond the Baskett benchmark , 1984, CARN.

[20]  Jack J. Dongarra,et al.  The LINPACK Benchmark: An Explanation , 1988, ICS.

[21]  M. L. Simmons,et al.  A close look at vector performance of register-to-register vector computers and a new model , 1987, SIGMETRICS '87.

[22]  Kai Hwang,et al.  Computer architecture and parallel processing , 1984, McGraw-Hill Series in computer organization and architecture.

[23]  S. I. Feldman,et al.  A portable Fortran 77 compiler , 1990 .

[24]  Peter J. Denning,et al.  An ACM executive committee position on the crisis in experimental computer science , 1979, CACM.

[25]  David S. Lindsay.  Methodology for Determining the Effects of Optimizing Compilers , 1986, Int. CMG Conference.

[26]  James W. Moore,et al.  A Benchmark Comparison of Three Supercomputers: Fujitsu VP-200, Hitachi S810/120, and Cray X-MP/2 , 1985, Computer.

[27]  Leonard J. Shustek,et al.  An instruction timing model of CPU performance , 1998, ISCA '98.

[28]  David A. Patterson.  A performance evaluation of the Intel 80286 , 1982, CARN.

[29]  Douglas W. Clark,et al.  A Characterization of Processor Performance in the vax-11/780 , 1984, ISCA '84.

[30]  Donald E. Knuth,et al.  An empirical study of FORTRAN programs , 1971, Softw. Pract. Exp..