Quantifying the Impact of Input Data Sets on Program Behavior and its Applications

Having a representative workload of the target domain of a microprocessor is extremely important throughout its design. The composition of a workload involves two issues: (i) which benchmarks to select and (ii) which input data sets to select per benchmark. Unfortunately, it is impossible to select a huge number of benchmarks and respective input sets due to the large instruction counts per benchmark and due to limitations on the available simulation time. In this paper, we use statistical data analysis techniques such as principal components analysis (PCA) and cluster analysis to ecien tly explore the workload space. Within this workload space, dieren t input data sets for a given benchmark can be displayed, a distance can be measured between program-input pairs that gives us an idea about their mutual behavioral dierences and representative input data sets can be selected for the given benchmark. This methodology is validated by showing that program-input pairs that are close to each other in this workload space indeed exhibit similar behavior. The nal goal is to select a limited set of representative benchmark-input pairs that span the complete workload space. Next to workload composition, we discuss two other possible applications, namely getting insight in the impact of input data sets on program behavior and evaluating the representativeness of sampled traces.

[1]  John Paul Shen,et al.  The intrinsic bandwidth requirements of ordinary programs , 1996, ASPLOS VII.

[2]  Kevin Skadron,et al.  Techniques for Accurate, Accelerated Processor Simulation: Analysis of Reduced Inputs and Sampling , 2002 .

[3]  Louise Trevillyan,et al.  Representative traces for processor models with infinite cache , 1996, Proceedings. Second International Symposium on High-Performance Computer Architecture.

[4]  Lieven Eeckhout,et al.  Workload design: selecting representative program-input pairs , 2002, Proceedings.International Conference on Parallel Architectures and Compilation Techniques.

[5]  Brad Calder,et al.  Automatically characterizing large scale program behavior , 2002, ASPLOS X.

[6]  S. McFarling.  Combining Branch Predictors , 1993 .

[7]  David W. Wall,et al.  Predicting program behavior using real or estimated profiles , 2004, SIGP.

[8]  Thomas M. Conte,et al.  Reducing state loss for effective trace sampling of superscalar processors , 1996, Proceedings International Conference on Computer Design. VLSI in Computers and Processors.

[9]  Michaela Smith.  Overcoming the challenges to feedback-directed optimization (Keynote Talk) , 2000, DYNAMO '00.

[10]  Wei-Chung Hsu,et al.  On the predictability of program behavior using different input data sets , 2002, Proceedings Sixth Annual Workshop on Interaction between Compilers and Computer Architectures.

[11]  Frederic T. Chong,et al.  HLS: combining statistical and symbolic simulation to guide microprocessor designs , 2000, Proceedings of 27th International Symposium on Computer Architecture (IEEE Cat. No.RS00201).

[12]  John Paul Shen,et al.  A framework for statistical modeling of superscalar processor performance , 1997, Proceedings Third International Symposium on High-Performance Computer Architecture.

[13]  Todd M. Austin,et al.  SimpleScalar: An Infrastructure for Computer System Modeling , 2002, Computer.

[14]  J. Boccio,et al.  Computer program manual , 1972 .

[15]  Lieven Eeckhout,et al.  Hybrid analytical-statistical modeling for efficiently exploring architecture and workload design spaces , 2001, Proceedings 2001 International Conference on Parallel Architectures and Compilation Techniques.

[16]  Douglas M. Hawkins,et al.  A statistically rigorous approach for improving simulation methodology , 2003, The Ninth International Symposium on High-Performance Computer Architecture, 2003. HPCA-9 2003. Proceedings..

[17]  J. Francis.  Statistica for Windows , 1995 .

[18]  Pradip Bose,et al.  Performance Analysis and Its Impact on Design , 1998, Computer.

[19]  Lizy K. John,et al.  Workload characterization: motivation, goals and methodology , 1998, Workload Characterization: Methodology and Case Studies. Based on the First Workshop on Workload Characterization.

[20]  John Flynn,et al.  Adapting the SPEC 2000 benchmark suite for simulation-based computer architecture research , 2001 .

[21]  André Seznec,et al.  Choosing representative slices of program execution for microarchitecture simulations: a preliminary , 2000 .

[22]  James E. Smith,et al.  Modeling superscalar processors via statistical simulation , 2001, Proceedings 2001 International Conference on Parallel Architectures and Compilation Techniques.

[23]  Brad Calder,et al.  Basic block distribution analysis to find periodic behavior and simulation points in applications , 2001, Proceedings 2001 International Conference on Parallel Architectures and Compilation Techniques.

[24]  David A. Wood,et al.  A Comparison of Trace-Sampling Techniques for Multi-Megabyte Caches , 1994, IEEE Trans. Computers.

[25]  Alan Jay Smith,et al.  Analysis of benchmark characteristics and benchmark performance prediction , 1996, TOCS.

[26]  Joseph A. Fisher,et al.  Predicting conditional branch directions from previous runs of a program , 1992, ASPLOS V.

[27]  B. Manly.  Multivariate Statistical Methods : A Primer , 1986 .

[28]  Gary Lauterbach.  Accelerating architectural simulation by parallel execution of trace samples , 1994, 1994 Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences.