Automated Performance Prediction for Scalable Parallel Computing

Abstract Performance prediction is necessary in order to deal with multi-dimensional performance effects on parallel systems. The compiler-generated analytical model developed in this paper accounts for the effects of cache behavior, CPU execution time and message passing overhead for real programs written in high level data-parallel languages. The performance prediction technique is shown to be effective in analyzing several non-trivial data-parallel applications as the problem size and number of processors vary. We leverage technology from the Maple symbolic manipulation system and the S-PLUS statistical package in order to present users with critical performance information necessary for performance debugging, architectural enhancement and procurement of parallel systems. The usability of these results is improved through specifying confidence intervals as well as predicted execution times for data-parallel applications.

[1]  Thomas Sterling,et al.  System software and tools for high performance computing environments , 1995 .

[2]  Geo. R. Lawrence Co..  Pacific Grove, California , 1906 .

[3]  G. C. Fox,et al.  What have we learnt from using real parallel machines to solve real problems? , 1989, C3P.

[4]  Thomas Fahringer,et al.  A static parameter based performance prediction tool for parallel programs , 1993, ICS '93.

[5]  M. J. Quinn,et al.  Analytical performance prediction on multicomputers , 1993, Supercomputing '93.

[6]  Thomas Fahringer.  Automatic Performance Prediction of Parallel Programs , 1996, Springer US.

[7]  Mark Crovella,et al.  The Search for Lost Cycles: A New Approach to Parallel Program Performance Evaluation , 1993 .

[8]  Trevor Hastie,et al.  Statistical Models in S , 1991 .

[9]  Manish Gupta,et al.  Demonstration of Automatic Data Partitioning Techniques for Parallelizing Compilers on Multicomputers , 1992, IEEE Trans. Parallel Distributed Syst..

[10]  Thomas Fahringer.  Toward symbolic performance prediction of parallel programs , 1996, Proceedings of International Conference on Parallel Processing.

[11]  Mark J. Clement,et al.  Performance prediction of PVM programs , 1996, Proceedings of International Conference on Parallel Processing.

[12]  Mark J. Clement,et al.  Multivariate statistical techniques for parallel performance prediction , 1995, Proceedings of the Twenty-Eighth Annual Hawaii International Conference on System Sciences.

[13]  Richard J. Harris.  A primer of multivariate statistics , 1975 .

[14]  Pankaj Mehra,et al.  Automated modeling of message-passing programs , 1994, Proceedings of International Workshop on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[15]  Philip J. Hatcher,et al.  Data-Parallel Programming on MIMD Computers , 1991, IEEE Trans. Parallel Distributed Syst..

[16]  Xian-He Sun,et al.  Scalability of Parallel Algorithm-Machine Combinations , 1994, IEEE Trans. Parallel Distributed Syst..

[17]  Franz Sötz,et al.  A Method for Performance Prediction of Parallel Programs , 1990, CONPAR.

[18]  Ken Kennedy,et al.  A static performance estimator to guide data partitioning decisions , 1991, PPOPP '91.

[19]  John L. Gustafson,et al.  Reevaluating Amdahl's law , 1988, CACM.

[20]  Allen D. Malony,et al.  Next-Generation Parallel Performance Visualization: A Prototyping Environment for Visualization Development , 1994, PARLE.

[21]  Mark J. Clement,et al.  Network Performance Modeling for PVM Clusters , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[22]  Mark J. Clement.  Analytical performance prediction of data-parallel programs , 1994 .