Validation methods for calibrating software effort models

COCONUT calibrates effort estimation models using an exhaustive search over the space of calibration parameters in a Cocomo I model. This technique is much simpler than other effort estimation method yet yields PRED levels comparable to those other methods. Also, it does so with less project data and fewer attributes (no scale factors). However, a comparison between COCONUT and other methods is complicated by differences in the experimental methods used for effort estimation. A review of those experimental methods concludes that software effort estimation models should be calibrated to local data using incremental holdout (not jack knife) studies, combined with randomization and hypothesis testing, repeated a statistically significant number of times.

[1]  Ian Witten,et al.  Data Mining , 2000 .

[2]  寺岡 寛,et al.  Engineering Economics , 2018, Nature.

[3]  Hamid Habibagahi,et al.  Estimating Software Productivity and Cost for NASA Projects , 1991 .

[4]  J. Hihn,et al.  Validation of spacecraft software cost estimation models for flight and ground systems , 2002 .

[5]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[6]  Chris F. Kemerer,et al.  An empirical validation of software cost estimation models , 1987, CACM.

[7]  Ellis Horowitz,et al.  Software Cost Estimation with COCOMO II , 2000 .

[8]  Gavin R. Finnie,et al.  Estimating software development effort with connectionist models , 1997, Inf. Softw. Technol..

[9]  Barry Boehm,et al.  The effects of software process maturity on software development effort , 1997 .

[10]  Martin J. Shepperd,et al.  Estimating Software Project Effort Using Analogies , 1997, IEEE Trans. Software Eng..

[11]  Gary D. Boetticher,et al.  21st-century AI: proud, not smug , 2003, IEEE Intelligent Systems.

[12]  Gary D. Boetticher When will it be done? Machine learner answers to the 300-billion-dollar question , 2003 .

[13]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[14]  David S. Christensen,et al.  Calibrating Software Cost Models to Department of Defense DatabasesA Review of Ten Studies , 1998 .

[15]  Barry W. Boehm,et al.  Bayesian Analysis of Empirical Software Engineering Cost Models , 1999, IEEE Trans. Software Eng..

[16]  Paul R. Cohen,et al.  Empirical methods for artificial intelligence , 1995, IEEE Expert.

[17]  Keith Phalp,et al.  An investigation of machine learning based prediction systems , 2000, J. Syst. Softw..

[18]  Douglas Fisher,et al.  Machine Learning Approaches to Estimating Software Development Effort , 1995, IEEE Trans. Software Eng..

[19]  F KemererChris An empirical validation of software cost estimation models , 1987 .

[20]  Sherry Stukes,et al.  Software Cost Model Calibration , 1998 .

[21]  Robert C. Holte,et al.  Very Simple Classification Rules Perform Well on Most Commonly Used Datasets , 1993, Machine Learning.

[22]  Barry Boehm,et al.  From Multiple Regression to Bayesian Analysis for Calibrating COCOMO II , 1999 .