Towards efficient supercomputing: searching for the right efficiency metric

Efficiency in supercomputing has traditionally focused on execution time. In early 2000's, the concept of total cost of ownership was re-introduced, with the introduction of efficiency measure to include aspects such as energy and space. Yet the supercomputing community has never agreed upon a metric that can cover these aspects completely and also provide a fair basis for comparison. This paper examines the metrics that have been proposed in the past decade, and proposes a vector-valued metric for efficient supercomputing. Using this metric, the paper presents a study of where the supercomputing industry has been and where it stands today with respect to efficient supercomputing.

[1]  Mark Horowitz,et al.  Energy dissipation in general purpose microprocessors , 1996, IEEE J. Solid State Circuits.

[2]  Wolfgang E. Nagel,et al.  Flexible workload generation for HPC cluster efficiency benchmarking , 2012, Computer Science - Research and Development.

[3]  Rong Ge,et al.  Improvement of power-performance efficiency for high-end computing , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[4]  Trevor N. Mudge,et al.  Challenges and Opportunities for Extremely Energy-Efficient Processors , 2010, IEEE Micro.

[5]  Stephen Berard,et al.  Implications of Historical Trends in the Electrical Efficiency of Computing , 2011, IEEE Annals of the History of Computing.

[6]  Jack J. Dongarra,et al.  The LINPACK Benchmark: past, present and future , 2003, Concurr. Comput. Pract. Exp..

[7]  Kushagra Vaid,et al.  Optimizing benchmark configurations for energy efficiency , 2011, ICPE '11.

[8]  Christoforos E. Kozyrakis,et al.  JouleSort: a balanced energy-efficiency benchmark , 2007, SIGMOD '07.

[9]  Tsugio Makimoto,et al.  The Cooler the Better: New Directions in the Nomadic Age , 2001, Computer.

[10]  Daniel Schall,et al.  Towards an Enhanced Benchmark Advocating Energy-Efficient Systems , 2011, TPCTC.

[11]  Lieven Eeckhout,et al.  SWEEP: evaluating computer system energy efficiency using synthetic workloads , 2011, HiPEAC.

[12]  Thomas F. Wenisch,et al.  Does low-power design imply energy efficiency for data centers? , 2011, IEEE/ACM International Symposium on Low Power Electronics and Design.

[13]  Erich Strohmaier.  Generalized utility metrics for supercomputers , 2009, Computer Science - Research and Development.

[14]  Vivek Sarkar,et al.  Software challenges in extreme scale systems , 2009 .

[15]  Matthias S. Müller,et al.  Quantifying power consumption variations of HPC systems using SPEC MPI benchmarks , 2010, Computer Science - Research and Development.

[16]  Lieven Eeckhout,et al.  Trends in Server Energy Proportionality , 2011, Computer.

[17]  Klaus-Dieter Lange,et al.  The design and development of the server efficiency rating tool (SERT) , 2011, ICPE '11.

[18]  Thomas M. Conte,et al.  Advances in Benchmarking Techniques: New Standards and Quantitative Metrics , 1995, Adv. Comput..

[19]  Kushagra Vaid,et al.  Energy benchmarks: a detailed analysis , 2010, e-Energy.

[20]  Wu-chun Feng,et al.  Towards efficient supercomputing: a quest for the right metric , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[21]  Manish Gupta,et al.  Power-Aware Microarchitecture: Design and Modeling Challenges for Next-Generation Microprocessors , 2000, IEEE Micro.

[22]  Paul I. Pénzes,et al.  Energy-delay efficiency of VLSI computations , 2002, GLSVLSI '02.

[23]  Matthias S. Müller,et al.  SPEC MPI2007—an application benchmark suite for parallel systems using MPI , 2010, Concurr. Comput. Pract. Exp..

[24]  Anil Kumar,et al.  Workload Characterization of the SPECpower_ssj2008 Benchmark , 2008, SIPEW.

[25]  Wu-chun Feng,et al.  Emerging Trends on the Evolving Green500: Year Three , 2011, 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum.

[26]  Constantine Bekas,et al.  A new energy aware performance metric , 2010, Computer Science - Research and Development.

[27]  Sandeep K. S. Gupta,et al.  Energy Proportionality and the Future: Metrics and Directions , 2010, 2010 39th International Conference on Parallel Processing Workshops.

[28]  Alain J. Martin.  Towards an energy complexity of computation , 2001, Inf. Process. Lett..

[29]  Arthur Howard,et al.  The State of Energy and Performance Benchmarking for Enterprise Servers , 2009, TPCTC.

[30]  Luiz André Barroso,et al.  The Case for Energy-Proportional Computing , 2007, Computer.