Using Information from Prior Runs to Improve Automated Tuning Systems

Active Harmony is an automated runtime performance tuning system. In this paper we describe a parameter prioritizing tool to help focus on those parameters that are performance critical. Historical data is also utilized to further speed up the tuning process. We first verify our proposed approaches with synthetic data and finally we verify all the improvements on a real cluster-based web service system. Taken together, these changes allow the Active Harmony system to reduce the time spent tuning from 35% up to 50% and at the same time, reduce the variation in performance while tuning.

[1]  Jesús Labarta,et al.  A Framework for Performance Modeling and Prediction , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[2]  Ray Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[3]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[4]  David A. Patterson,et al.  A new approach to I/O performance evaluation: self-scaling I/O benchmarks, predicted I/O performance , 1993, SIGMETRICS '93.

[5]  Ruoming Jin,et al.  Performance prediction for random write reductions: a case study in modeling shared memory programs , 2002, SIGMETRICS '02.

[6]  I-Hsin Chung,et al.  Active Harmony: Towards Automated Performance Tuning , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[7]  R. Roehrkasse.  Linear programming in operations research , 1990, IEEE Potentials.

[8]  Cosimo Anglano,et al.  Predicting parallel applications performance on non-dedicated cluster platforms , 1998, ICS '98.

[9]  Matthias Schumann.  Automatic performance prediction to support cross development of parallel programs , 1996, SPDT '96.

[10]  Laura Carrington,et al.  A Framework for Application Performance Modeling and Prediction , 2002 .

[11]  Thomas Phan,et al.  Performance prediction of large parallel applications using parallel simulations , 1999, PPoPP '99.

[12]  Alan Jay Smith,et al.  Analysis of benchmark characteristics and benchmark performance prediction , 1996, TOCS.

[13]  I-Hsin Chung,et al.  Automated cluster-based Web service performance tuning , 2004, Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004..

[14]  R.J. Block,et al.  Automated Performance Prediction of Message-Passing Parallel Programs , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[15]  F. Berman,et al.  Adaptive Performance Prediction for Distributed Data-Intensive Applications , 1999, ACM/IEEE SC 1999 Conference (SC'99).

[16]  D. Anderson,et al.  Algorithms for minimization without derivatives , 1974 .

[17]  Wentong Cai,et al.  Performance prediction tools for parallel discrete-event simulation , 1999, Proceedings Thirteenth Workshop on Parallel and Distributed Simulation. PADS 99. (Cat. No.PR00155).

[18]  Thomas Fahringer.  Automatic Performance Prediction of Parallel Programs , 1996, Springer US.

[19]  Jeffrey K. Hollingsworth,et al.  Exposing application alternatives , 1999, Proceedings. 19th IEEE International Conference on Distributed Computing Systems (Cat. No.99CB37003).

[20]  R. Plackett,et al.  THE DESIGN OF OPTIMUM MULTIFACTORIAL EXPERIMENTS , 1946 .

[21]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[22]  Carla E. Brodley,et al.  Predictive application-performance modeling in a computational grid environment , 1999, Proceedings. The Eighth International Symposium on High Performance Distributed Computing (Cat. No.99TH8469).

[23]  Nesa L'abbe Wu,et al.  Linear programming and extensions , 1981 .

[24]  Jeffrey K. Hollingsworth,et al.  Prediction and adaptation in Active Harmony , 2004, Cluster Computing.

[25]  Daniel A. Reed,et al.  Performance Contracts: Predicting and Monitoring Grid Application Behavior , 2001, GRID.