The effect of profile choice and profile gathering methods on profile-driven optimization systems

Profile-driven optimization can produce substantial improvements in the quality of code produced by a compiler or link-time optimizer. In this work, we analyze several important aspects of profile-driven optimization. We examine the effectiveness of profile-driven optimization in two commercial-quality optimizers (Digital's GEM compiler and the link-time optimizer ‘alto’). We perform analyses to determine how much variability in profile-driven optimization performance results from choosing different training profiles, and to determine how much optimization benefit results from choosing more ‘accurate’ profiles (that is, profiles that better predict the way that a program is actually run). We examine low-overhead profiling methods such as static estimation (estimating profiles using static heuristics) and statistical sampling (gathering profiles by sampling only a small number of basic block executions). We analyze some profile-driven optimization results in great detail, and show a methodology for accounting for the profile-driven optimization effects of profile data associated with individual functions. Our results show that profile-driven optimization is effective on average, but unreliable when considering any individual benchmark. Using more accurate profiles is only weakly connected to improved profile-driven optimization performance for most benchmarks. However, low-overhead profiling techniques result in substantial degradations in the reliability and average performance of profile-driven optimization, often to the point of rendering the entire profile-driven optimization process useless. Our analysis also shows that the effects of profile-driven optimization are highly concentrated in the profile data associated with a few functions. Whether profile data improves or worsens the performance of optimized code, it is often possible to attribute the vast majority of this effect to the profile data associated with just a few functions.

[1]  Phil Winterbottom.  ACID: A Debugger Built From A Language , 1994, USENIX Winter.

[2]  Saumya K. Debray,et al.  alto: A Link-Time Optimizer for the DEC Alpha , 1996 .

[3]  James R. Larus,et al.  Branch prediction for free , 1993, PLDI '93.

[4]  Raymond H. Myers,et al.  Probability and Statistics for Engineers and Scientists. , 1973 .

[5]  Lieven Eeckhout,et al.  Workload design: selecting representative program-input pairs , 2002, Proceedings.International Conference on Parallel Architectures and Compilation Techniques.

[6]  Michael D. Smith,et al.  Overcoming the Challenges to Feedback-Directed Optimization , 2000, Dynamo.

[7]  Zheng Wang,et al.  A Statistical Analysis of User-Specific Profiles , 1998 .

[8]  Jeffrey Dean,et al.  ProfileMe: hardware support for instruction-level profiling on out-of-order processors , 1997, Proceedings of 30th Annual International Symposium on Microarchitecture.

[9]  P. Geoffrey Lowney,et al.  Feedback directed optimization in Compaq's compilation tools for Alpha , 1999 .

[10]  Michael A. Harrison,et al.  Accurate static estimators for program optimization , 1994, PLDI '94.

[11]  David W. Wall,et al.  Predicting program behavior using real or estimated profiles , 2004, SIGP.

[12]  Serap A. Savari,et al.  Comparing and Combining Profiles , 2000, J. Instr. Level Parallelism.

[13]  Lance M. Berc,et al.  Continuous profiling: where have all the cycles gone? , 1997, ACM Trans. Comput. Syst..

[14]  Joseph A. Fisher,et al.  Predicting conditional branch directions from previous runs of a program , 1992, ASPLOS V.

[15]  Wen-mei W. Hwu,et al.  Speculative hedge: regulating compile-time speculation against profile variations , 1996, Proceedings of the 29th Annual IEEE/ACM International Symposium on Microarchitecture. MICRO 29.

[16]  Zheng Wang,et al.  System support for automatic profiling and optimization , 1997, SOSP.

[17]  James R. Larus,et al.  Static branch frequency and program profile analysis , 1994, MICRO 27.

[18]  Zheng Wang,et al.  Profile-Based Optimization with Statistical Profiles , 1997 .