Software project economics: a roadmap

The objective of this paper is to consider research progress in the field of software project economics with a view to identifying important challenges and promising research directions. I argue that this is an important sub-discipline since this will underpin any cost-benefit analysis used to justify the resourcing, or otherwise, of a software project. To accomplish this I conducted a bibliometric analysis of peer reviewed research articles to identify major areas of activity. My results indicate that the primary goal of more accurate cost prediction systems remains largely unachieved. However, there are a number of new and promising avenues of research including: how we can combine results from primary studies, integration of multiple predictions and applying greater emphasis upon the human aspects of prediction tasks. I conclude that the field is likely to remain very challenging due to the people-centric nature of software engineering, since it is in essence a design task. Nevertheless the need for good economic models will grow rather than diminish as software becomes increasingly ubiquitous.

[1]  Chris Abts,et al.  COCOTS: A COTS Software Integration Lifecycle Cost Model - Model Overview and Preliminary Data Collection Findings , 2000 .

[2]  Barbara A. Kitchenham,et al.  Estimates, Uncertainty, and Risk , 1997, IEEE Softw..

[3]  Tim Menzies,et al.  The \{PROMISE\} Repository of Software Engineering Databases. , 2005 .

[4]  Stephen G. MacDonell,et al.  Combining techniques to optimize effort predictions in software project management , 2003, J. Syst. Softw..

[5]  D. Ross Jeffery,et al.  Calibrating estimation tools for software development , 1990, Softw. Eng. J..

[6]  Soumitra Dutta,et al.  Performance Evaluation of General and Company Specific Models in Software Development Effort Estimation , 1999 .

[7]  David W. Aha,et al.  A Comparative Evaluation of Sequential Feature Selection Algorithms , 1995, AISTATS.

[8]  Kari Känsälä,et al.  Inter-item correlations among function points , 1993, [1993] Proceedings First International Software Metrics Symposium.

[9]  Magne Jørgensen,et al.  An analysis of data sets used to train and validate cost prediction systems , 2005, PROMISE '05.

[10]  Michael A. Cusumano,et al.  A quantitative analysis of U.S. and Japanese practice and performance in software development , 1990 .

[11]  Jean-Marc Desharnais,et al.  A comparison of software effort estimation techniques: Using function points with neural networks, case-based reasoning and regression models , 1997, J. Syst. Softw..

[12]  Keith Phalp,et al.  An investigation of machine learning based prediction systems , 2000, J. Syst. Softw..

[13]  Martin J. Shepperd,et al.  Making inferences with small numbers of training sets , 2002, IEE Proc. Softw..

[14]  Albert L. Lederer,et al.  Software management and cost estimating error , 2000, J. Syst. Softw..

[15]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[16]  D. Altman,et al.  Measuring inconsistency in meta-analyses , 2003, BMJ : British Medical Journal.

[17]  Rajiv D. Banker,et al.  Scale Economies in New Software Development , 2013, IEEE Transactions on Software Engineering.

[18]  H. D. Benington,et al.  Production of Large Computer Programs , 1983, Annals of the History of Computing.

[19]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[20]  Chris F. Kemerer,et al.  Reliability of function points measurement: a field experiment , 2015, CACM.

[21]  D. Ross Jeffery,et al.  Function Points in the Estimation and Evaluation of the Software Process , 1990, IEEE Trans. Software Eng..

[22]  David J. Hand,et al.  How to lie with bad data , 2005 .

[23]  Michael A. Cusumano,et al.  Software Development Worldwide: The State of the Practice , 2003, IEEE Softw..

[24]  Chris F. Kemerer,et al.  An empirical validation of software cost estimation models , 1987, CACM.

[25]  Ellis Horowitz,et al.  Software Cost Estimation with COCOMO II , 2000 .

[26]  Magne Jørgensen,et al.  Better sure than safe? Over-confidence in judgement based software development effort prediction intervals , 2004, J. Syst. Softw..

[27]  Guilherme Horta Travassos,et al.  A systematic review of cross- vs. within- company cost estimation studies , 2006 .

[28]  Barbara A. Kitchenham,et al.  The question of scale economies in software - why cannot researchers agree? , 2002, Inf. Softw. Technol..

[29]  T. Wright,et al.  Organizational Benchmarking Using the ISBSG Data Repository , 2001, IEEE Softw..

[30]  Ioannis Stamelos,et al.  Software productivity and effort prediction with ordinal regression , 2005, Inf. Softw. Technol..

[31]  Khaled El Emam,et al.  Software Cost Estimation with Incomplete Data , 2001, IEEE Trans. Software Eng..

[32]  Stuart E. Madnick,et al.  Special Feature: Impact of Schedule Estimation on Software Project Behavior , 1986, IEEE Software.

[33]  Alain Abran,et al.  Function Points Analysis: An Empirical Study of Its Measurement Processes , 1996, IEEE Trans. Software Eng..

[34]  Stephen G. MacDonell,et al.  What accuracy statistics really measure , 2001, IEE Proc. Softw..

[35]  Robert T. Hughes,et al.  Expert judgement as an estimating method , 1996, Inf. Softw. Technol..

[36]  Huan Liu,et al.  Book review: Machine Learning, Neural and Statistical Classification Edited by D. Michie, D.J. Spiegelhalter and C.C. Taylor (Ellis Horwood Limited, 1994) , 1996, SGAR.

[37]  Charles R. Symons,et al.  Function Point Analysis: Difficulties and Improvements , 1988, IEEE Trans. Software Eng..

[38]  Carolyn Mair,et al.  The consistency of empirical comparisons of regression and analogy-based software project cost prediction , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[39]  Michelle Cartwright,et al.  Predicting with Sparse Data , 2001, IEEE Trans. Software Eng..

[40]  Tore Dybå,et al.  Evidence-based software engineering , 2004, Proceedings. 26th International Conference on Software Engineering.

[41]  José Javier Dolado,et al.  On the problem of the software cost function , 2001, Inf. Softw. Technol..

[42]  John E. Gaffney,et al.  Software Function, Source Lines of Code, and Development Effort Prediction: A Software Science Validation , 1983, IEEE Transactions on Software Engineering.

[43]  AbranAlain,et al.  Function Points Analysis , 1996 .

[44]  Magne Jørgensen,et al.  An effort prediction interval approach based on the empirical distribution of previous estimation accuracy , 2003, Inf. Softw. Technol..

[45]  Y. Miyazaki,et al.  Robust regression for developing software estimation models , 1994, J. Syst. Softw..

[46]  Martin J. Shepperd,et al.  An experiment on software project size and effort estimation , 2003, 2003 International Symposium on Empirical Software Engineering, 2003. ISESE 2003. Proceedings..

[47]  Martin J. Shepperd,et al.  Estimating Software Project Effort Using Analogies , 1997, IEEE Trans. Software Eng..

[48]  Barbara A. Kitchenham,et al.  Empirical studies of assumptions that underlie software cost-estimation models , 1992, Inf. Softw. Technol..

[49]  Ronald Gulezian Reformulating and calibrating COCOMO , 1991, J. Syst. Softw..

[50]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007, IEEE Transactions on Software Engineering.

[51]  Martin Shepperd,et al.  Case and Feature Subset Selection in Case-Based Software Project Effort Prediction , 2003 .

[52]  Martin J. Shepperd,et al.  Comparing Software Prediction Techniques Using Simulation , 2001, IEEE Trans. Software Eng..

[53]  Rajiv D. Banker,et al.  Evidence on economies of scale in software development , 1994, Inf. Softw. Technol..

[54]  D. Ross Jeffery,et al.  A Comparison of Function Point Counting Techniques , 1993, IEEE Trans. Software Eng..

[55]  Martin J. Shepperd,et al.  Search Heuristics, Case-based Reasoning And Software Project Effort Prediction , 2002, GECCO.

[56]  Computer Staff,et al.  Function Points , 1994 .

[57]  Magne Jørgensen,et al.  A review of studies on expert estimation of software development effort , 2004, J. Syst. Softw..

[58]  Karen T. Lum,et al.  Selecting Best Practices for Effort Estimation , 2006, IEEE Transactions on Software Engineering.