An empirical study of experience-based software defect content estimation methods

Capture-recapture models and curve-fitting models have been proposed to estimate the remaining number of defects after a review. This estimation gives valuable information to monitor and control software reliability. However, the different models provide different estimates making it difficult to know which estimate is the most accurate. One possible solution is to, as in this paper, focus on different opportunities to estimate intervals. The study is based on thirty capture-recapture data sets from software reviews. Twenty of the data sets are used to create different models to perform estimation. The models are then evaluated on the remaining ten data sets. The study shows that the use of historical data in model building is one way to overcome some of the problems experienced with both capture-recapture and curve-fitting models, to estimate the defect content after a review.

[1]  Harvey P. Siy,et al.  An experiment to assess the cost-benefits of code inspections in large scale software development , 1995, SIGSOFT '95.

[2]  Lionel C. Briand,et al.  Quantitative evaluation of capture-recapture models to control software inspections , 1997, Proceedings The Eighth International Symposium on Software Reliability Engineering.

[3]  Stephen G. Eick,et al.  Estimating software fault content before coding , 1992, International Conference on Software Engineering.

[4]  Claes Wohlin,et al.  Evaluation of using Capture-Recapture Methods on Software Review Data , 1999 .

[5]  Lionel C. Briand,et al.  A comparison and integration of capture-recapture models and the detection profile method , 1998, Proceedings Ninth International Symposium on Software Reliability Engineering (Cat. No.98TB100257).

[6]  David R. Anderson,et al.  Statistical inference from capture data on closed animal populations , 1980 .

[7]  Niclas Ohlsson,et al.  Predicting Fault-Prone Software Modules in Telephone Switches , 1996, IEEE Trans. Software Eng..

[8]  Lionel C. Briand,et al.  A Comprehensive Evaluation of Capture-Recapture Models for Estimating Software Defect Content , 2000, IEEE Trans. Software Eng..

[9]  Harvey P. Siy,et al.  An Experiment ot Assess the Cost-Benefits of Code Inspections in Large Scale Software Development , 1997, IEEE Trans. Software Eng..

[10]  A Chao,et al.  Estimating population size for capture-recapture data when capture probabilities vary by time and individual animal. , 1992, Biometrics.

[11]  Claes Wohlin,et al.  An experimental evaluation of capture‐recapture in software inspections , 1995, Softw. Test. Verification Reliab..

[12]  Michael Fagan Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[13]  Michael E. Fagan Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[14]  Claes Wohlin,et al.  Defect content estimations from review data , 1998, Proceedings of the 20th International Conference on Software Engineering.