Quantifying hardware counter sampling error in computer system workload characterization

This technical report explores workload characterization using processor hardware counter sampling. We assume that we are measuring more hardware counter events than the number of physical counters on the processor, i.e., the counters are set to measure different events after every measurement. We characterize workloads on typical timescales between 5 and 30 minutes, with different phasing properties. We evaluate two competing strategies, a short sample time strategy designed to minimize overhead and a long sample time strategy designed to get better averaging. We find that both strategies are likely to produce accurate results, independent of underlying workload phasing, but neither strategy is ideal. We find that the optimal strategy is a fairly short sample time combined with continuous iteration over the counter set.

[1]  Subhasis Banerjee,et al.  Exploiting program execution phases to trade power and performance for media workload , 2004, ASP-DAC 2004: Asia and South Pacific Design Automation Conference 2004 (IEEE Cat. No.04EX753).

[2]  Richard McDougall,et al.  Solaris Internals: Solaris 10 and OpenSolaris Kernel Architecture , 2006 .

[3]  Brad Calder,et al.  Automatically characterizing large scale program behavior , 2002, ASPLOS X.

[4]  W. Cleveland.  Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .

[5]  Brad Calder,et al.  Structures for phase classification , 2004, IEEE International Symposium on - ISPASS Performance Analysis of Systems and Software, 2004.

[6]  Brad Calder,et al.  Phase tracking and prediction , 2003, ISCA '03.

[7]  Brad Calder,et al.  Basic block distribution analysis to find periodic behavior and simulation points in applications , 2001, Proceedings 2001 International Conference on Parallel Architectures and Compilation Techniques.

[8]  Daniel A. Menascé.  Web Server Software Architectures , 2003, IEEE Internet Comput..

[9]  Lee W. Hoevel,et al.  The Software-Cache Connection , 1981, IBM J. Res. Dev..

[10]  Sandhya Dwarkadas,et al.  Characterizing and predicting program behavior and its variability , 2003, 2003 12th International Conference on Parallel Architectures and Compilation Techniques.

[11]  W. Cleveland.  LOWESS: A Program for Smoothing Scatterplots by Robust Locally Weighted Regression , 1981 .

[12]  Mark Crovella,et al.  Computer Systems Performance Evaluation , 2007 .

[13]  Jeanine Cook,et al.  Examining performance differences in workload execution phases , 2001 .

[14]  Ashok K. Agrawala,et al.  An Approach to the Workload Characterization Problem , 1976, Computer.