An Analysis of Accuracy and Learning in Software Project Estimating

This paper presents a study into the accuracy of different dimensions of IT project estimating: schedule, budget & effort. The study is based on a dataset of 171 projects that have been collected at the IT-department of a large Dutch multinational company. The paper also analyses whether there is any learning (improvement) effect over time. Our results show that there is no relation between accuracy of budget, schedule and effort in the analyzed organization. Besides, they show that over time there is no change of the accuracy effectiveness and efficiency. The paper reflects and provides recommendations on how to improve the learning from historical estimates.

[1]  Chris F. Kemerer,et al.  An empirical validation of software cost estimation models , 1987, CACM.

[2]  Kjetil Moløkken-Østvold,et al.  A survey on software estimation in the Norwegian industry , 2004, 10th International Symposium on Software Metrics, 2004. Proceedings..

[3]  Barry Boehm,et al.  Software Cost Estimation with Cocomo II with Cdrom , 2000 .

[4]  Dar-Biau Liu,et al.  Metrics for requirements engineering , 1995, J. Syst. Softw..

[5]  Stefan Koch,et al.  Software project effort estimation with voting rules , 2009, Decis. Support Syst..

[6]  Luiz A. Laranjeira.  Software Size Estimation of Object-Oriented Systems , 1990, IEEE Trans. Software Eng..

[7]  Magne Jørgensen,et al.  Reasons for software effort estimation error: impact of respondent role, information collection approach, and data analysis method , 2004, IEEE Transactions on Software Engineering.

[8]  Michael J. Prietula,et al.  Examining the Feasibility of a Case-Based Reasoning Model for Software Effort Estimation , 1992, MIS Q..

[9]  Michiel van Genuchten,et al.  Why is Software Late? An Empirical Study of Reasons For Delay in Software Development , 1991, IEEE Trans. Software Eng..

[10]  Steve McConnell.  Software Estimation: Demystifying the Black Art , 2006 .

[11]  Magne Jørgensen,et al.  An effort prediction interval approach based on the empirical distribution of previous estimation accuracy , 2003, Inf. Softw. Technol..

[12]  Chris Verhoef,et al.  The rise and fall of the Chaos report figures , 2010, IEEE Software.

[13]  Chris Verhoef,et al.  Quantifying IT forecast quality , 2009, Sci. Comput. Program..

[14]  Stephen G. MacDonell,et al.  Combining techniques to optimize effort predictions in software project management , 2003, J. Syst. Softw..

[15]  Allen S. Parrish,et al.  An Empirical Study Using Task Assignment Patterns to Improve the Accuracy of Software Effort Estimation , 2001, IEEE Trans. Software Eng..

[16]  Stuart E. Madnick,et al.  On the portability of quantitative software estimation models , 2015, Inf. Manag..

[17]  Magne Jørgensen,et al.  Forecasting of software development work effort: Evidence on expert judgement and formal models , 2007 .

[18]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[19]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[20]  van GenuchtenMichiel.  Why is Software Late? An Empirical Study of Reasons for Delay in Software Development , 1991 .

[21]  Magne Jørgensen,et al.  A comparison of software project overruns - flexible versus sequential development models , 2005, IEEE Transactions on Software Engineering.

[22]  F. Ashcroft,et al.  VIII. References , 1955 .

[23]  Stephen G. MacDonell,et al.  What accuracy statistics really measure , 2001, IEE Proc. Softw..