Defining a catalog of indicators to support process performance analysis

To achieve high-level maturity in software process improvement, organizations must change the project management focus from the empirical assessment of process performance to quantitative management of the software process based on performance measures and statistical techniques. To this purpose, managers need to work with years of measurement data to establish control limits and conduct performance analyses. In this context, some difficulties are encountered: Which measures should be used, considering that early definition is important? Which statistical technique is more adequate in each case? How should we work with the data? We found more than 500 different measures in the literature applicable to software process performance and a number of different statistical techniques to analyze and choose from. This article aims at defining a catalog of indicators with their related measures in answer to the above questions, in a way that can help managers perform process performance analysis of the Capability Maturity Model Integration for Development (CMMI-DEV) engineering processes. This article presents how we define and use this catalog and the complete specification of two indicators. Copyright © 2009 John Wiley & Sons, Ltd.

[1]  David Herron,et al.  Function Point Analysis: Measurement Practices for Successful Software Projects , 2000 .

[2]  Linda H. Rosenberg,et al.  Metrics in software process assessment, quality assurance and risk assessment , 1994, Proceedings of 1994 IEEE 2nd International Software Metrics Symposium.

[3]  Daniel Galin,et al.  Do SQA programs work - CMM works. a meta analysis , 2005, IEEE International Conference on Software - Science, Technology & Engineering (SwSTE'05).

[4]  Duncan Dubugras Alcoba Ruiz,et al.  SPDW: A Software Development Process Performance Data Warehousing Environment , 2006, 2006 30th Annual IEEE/NASA Software Engineering Workshop.

[5]  Annie Combelles,et al.  A quantitative approach to software process management , 1994, Proceedings of 1994 IEEE 2nd International Software Metrics Symposium.

[6]  Kent Johnson,et al.  Interpreting the CMMI : A Process Improvement Approach, Second Edition , 2008 .

[7]  AgrawalManish,et al.  Software Effort, Quality, and Cycle Time , 2007 .

[8]  Donald J. Wheeler,et al.  Understanding Statistical Process Control , 1986 .

[9]  Yuan Zhang,et al.  Research on CMMI-based Software Process Metrics , 2006, First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06).

[10]  John McGarry.  Practical Software Measurement: Objective Information for Decision Makers , 2001 .

[11]  Tosiyasu L. Kunii,et al.  Software Metrics Knowledge and Databases for Project Management , 1999, IEEE Trans. Knowl. Data Eng..

[12]  Patrik Berander,et al.  A goal question metric based approach for efficient measurement framework definition , 2006, ISESE '06.

[13]  N. Eickelmann,et al.  Statistical Process Control: What You Don't Measure Can Hurt You! , 2003, IEEE Softw..

[14]  Robert E. Park,et al.  Software Size Measurement: A Framework for Counting Source Statements , 1992 .

[15]  D. Ross Jeffery,et al.  Lessons learnt from the analysis of large-scale corporate databases , 2006, ICSE.

[16]  William A. Florac,et al.  Measuring the Software Process: Statistical Process Control for Software Process Improvement , 1999 .

[17]  William W. Agresti.  Lightweight Software Metrics: The P10 Framework , 2006, IT Professional.

[18]  Manish Agrawal,et al.  Software Effort, Quality, and Cycle Time: A Study of CMM Level 5 Projects , 2007, IEEE Transactions on Software Engineering.

[19]  Katrina Maxwell.  Collecting Data for Comparability: Benchmarking Software Development Productivity , 2001, IEEE Softw..

[20]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[21]  Norman E. Fenton,et al.  Implementing Effective Software Metrics Programs , 1997, IEEE Softw..

[22]  Frank E. McGarry,et al.  Measuring the impacts individual process maturity attributes have on software products , 1998, Proceedings Fifth International Software Metrics Symposium. Metrics (Cat. No.98TB100262).

[23]  Christophe Debou,et al.  Decision making for software process improvement: A quantitative approach , 1994, J. Syst. Softw..

[24]  Philippe Kruchten,et al.  The Rational Unified Process Made Easy - A Practitioner's Guide to the RUP , 2003, Addison Wesley object technology series.

[25]  Daniel Galin,et al.  Are CMM Program Investments Beneficial? Analyzing Past Studies , 2006, IEEE Software.

[26]  Mayuram S. Krishnan,et al.  The impact of institutional forces on software metrics programs , 2005, IEEE Transactions on Software Engineering.

[27]  Forrest Shull,et al.  Using the ISO/IEC 9126 product quality model to classify defects: A controlled experiment , 2012, EASE.

[28]  James D. Herbsleb,et al.  Conceptual simplicity meets organizational complexity: case study of a corporate metrics program , 1998, Proceedings of the 20th International Conference on Software Engineering.

[29]  Silvio Romero de Lemos Meira,et al.  A comparative study on software reuse metrics and economic models from a traceability perspective , 2005, IRI -2005 IEEE International Conference on Information Reuse and Integration, Conf, 2005..

[30]  E. Chikofsky,et al.  Using metrics to justify investment in IT , 1999 .

[31]  Qing Wang,et al.  Measuring and improving software process in China , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[32]  Daniel Galin,et al.  Benefits of a Higher Quality Level of the Software Process: Two Organizations Compared , 2007 .