Company-Wide Implementation of Metrics for Early Software Fault Detection

To shorten time-to-market and improve customer satisfaction, software development companies commonly want to use metrics for assessing and improving the performance of their development projects. This paper describes a measurement concept for assessing how good an organization is at finding faults when most cost-effective, i. e. in most cases early. The paper provides results and lessons learned from applying the measurement concept widely at a large software development company. A major finding was that on average, 64 percent of all faults found would have been more cost effective to find during unit tests. An in-depth study of a few projects at a development unit also demonstrated how to use the measurement concept for identifying which parts in the fault detection process that needs to be improved to become more efficient (e.g. reduce the amount of time spent on rework).

[1]  J. J. Whelan,et al.  5th international conference on software engineering , 1981, SOEN.

[2]  Lars Lundberg,et al.  Results from introducing component-level test automation and Test-Driven Development , 2006, J. Syst. Softw..

[3]  Ellis Horowitz,et al.  Software Cost Estimation with COCOMO II , 2000 .

[4]  RunesonPer,et al.  What Do We Know about Defect Detection Methods , 2006 .

[5]  Ram Chillarege,et al.  Test and development process retrospective - a case study using ODC triggers , 2002, Proceedings International Conference on Dependable Systems and Networks.

[6]  Barry Boehm,et al.  Top 10 list [software development] , 2001 .

[7]  Charles R. McClure,et al.  Measuring and managing performance in organizations , 1997 .

[8]  Mayuram S. Krishnan,et al.  Measurement Programs in Software Development: Determinants of Success , 2002, IEEE Trans. Software Eng..

[9]  Michael Daskalantonakis,et al.  A Practical View of Software Measurement and Implementation Experiences Within Motorola , 1992, IEEE Trans. Software Eng..

[10]  E. Ziegel,et al.  The Balanced Scorecard , 1998 .

[11]  Claes Wohlin,et al.  Faults-slip-through - a concept for measuring the efficiency of the test process , 2006, Softw. Process. Improv. Pract..

[12]  Barry W. Boehm,et al.  Software Defect Reduction Top 10 List , 2001, Computer.

[13]  David N. Card,et al.  Learning from Our Mistakes with Defect Causal Analysis , 1999, IEEE Softw..

[14]  Alan R. Hevner,et al.  Phase containment metrics for software quality improvement , 1999, Inf. Softw. Technol..

[15]  Robert B. Grady,et al.  Practical Software Metrics for Project Management and Process Improvement , 1992 .

[16]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[17]  Victor R. Basili,et al.  Validation on an Approach for Improving Existing Measurement Frameworks , 2000, IEEE Trans. Software Eng..

[18]  Thomas Thelin,et al.  An industrial case study of the verification and validation activities , 2003, Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717).

[19]  Austen Rainer,et al.  Key success factors for implementing software process improvement: a maturity-based analysis , 2002, J. Syst. Softw..

[20]  Carol A. Dekkers,et al.  Steer Clear of Hazards on the Road to Software Measurement Success , 2004 .

[21]  N CardDavid.  Learning from Our Mistakes with Defect Causal Analysis , 1998 .

[22]  Shari Lawrence Pfleeger,et al.  Lessons learned in building a corporate metrics program , 1993, IEEE Software.

[23]  Tracy Hall,et al.  De-motivators for software process improvement: an analysis of practitioners' views , 2003, J. Syst. Softw..

[24]  Stephen H. Kan,et al.  Metrics and Models in Software Quality Engineering , 1994, SOEN.

[25]  Colin Potts,et al.  Software-engineering research revisited , 1993, IEEE Software.

[26]  E. Ziegel,et al.  Balanced Scorecard , 2019, Encyclopedia of Public Administration and Public Policy, Third Edition.

[27]  Barry W. Boehm,et al.  What we have learned about fighting defects , 2002, Proceedings Eighth IEEE Symposium on Software Metrics.

[28]  Steven R. Rakitin,et al.  Software verification and validation for practitioners and managers , 2001 .