Measuring HPC productivity

One key to improving high-performance computing (HPC) productivity is finding better ways to measure it. We define productivity in terms of mission goals, i.e., greater productivity means that more science is accomplished with less cost and effort. Traditional software productivity metrics and computing benchmarks have proven inadequate for assessing or predicting such end-to-end productivity. In this paper we introduce a new approach to measuring productivity in HPC applications that addresses both development time and execution time. Our goal is to develop a public repository of effective productivity benchmarks that anyone in the HPC community can apply to assess or predict productivity.

[1]  D. L. Parnas,et al.  On the criteria to be used in decomposing systems into modules , 1972, Software Pioneers.

[2]  Frederick P. Brooks,et al.  The Mythical Man-Month: Essays on Softw , 1978 .

[3]  David A. Bader,et al.  A Framework for Measuring Supercomputer Productivity , 2004, Int. J. High Perform. Comput. Appl..

[4]  Barbara A. Kitchenham,et al.  Empirical studies of assumptions that underlie software cost-estimation models , 1992, Inf. Softw. Technol..

[5]  P. Kidwell,et al.  The mythical man-month: Essays on software engineering , 1996, IEEE Annals of the History of Computing.

[6]  Adam A. Porter,et al.  Comparing Detection Methods for Software Requirements Inspections: A Replicated Experiment , 1995, IEEE Trans. Software Eng..

[7]  Gerald M. Weinberg,et al.  Goals and Performance in Computer Programming , 1974 .

[8]  Dewayne E. Perry,et al.  Understanding and Improving Time Usage in Software Development , 1995 .

[9]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[10]  Barbara Farbey,et al.  Exploring Requirements: Quality Before Design , 1991 .

[11]  J. Gustafson Purpose-Based Benchmarks , 2004, Int. J. High Perform. Comput. Appl..

[12]  Fred P. Brooks,et al.  The Mythical Man-Month , 1975, Reliable Software.

[13]  Philip M. Johnson,et al.  Beyond the Personal Software Process: Metrics collection and analysis for the differently disciplined , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[14]  John E. Gaffney,et al.  Software Function, Source Lines of Code, and Development Effort Prediction: A Software Science Validation , 1983, IEEE Transactions on Software Engineering.

[15]  Douglass E. Post,et al.  Software Project Management and Quality Engineering Practices for Complex, Coupled Multiphysics, Massively Parallel Computational Simulations: Lessons Learned From ASCI , 2004, Int. J. High Perform. Comput. Appl..

[16]  Jeremy Kepner HPC Productivity: An Overarching View , 2004, Int. J. High Perform. Comput. Appl..

[17]  Michael E. Fagan Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[18]  Capers Jones,et al.  Programming Productivity , 1986 .

[19]  Barry W. Boehm,et al.  Software cost modeling: Some lessons learned , 1980, J. Syst. Softw..

[20]  Barry W. Boehm,et al.  Cost models for future software life cycle processes: COCOMO 2.0 , 1995, Ann. Softw. Eng..