A comparison of development effort estimation techniques for Web hypermedia applications

Several studies have compared the prediction accuracy of different types of techniques with emphasis placed on linear and stepwise regressions, and case-based reasoning (CBR). We believe the use of only one type of CBR technique may bias the results, as there are others that can also be used for effort prediction. This paper has two objectives. The first is to compare the prediction accuracy of three CBR techniques to estimate the effort to develop Web hypermedia applications. The second objective is to compare the prediction accuracy of the best CBR technique, according to our findings, against three commonly used prediction models, namely multiple linear regression, stepwise regression and regression trees. One dataset was used in the estimation process and the results showed that different measures of prediction accuracy gave different results. MMRE and MdMRE showed better prediction accuracy for multiple regression models whereas box plots showed better accuracy for CBR.

[1]  Stephen G. MacDonell,et al.  A comparison of model building techniques to develop predictive equations for software metrics , 1997 .

[2]  Martin Shepperd,et al.  Experiences Using Case-Based Reasoning to Predict Software Project Effort , 2000 .

[3]  Barbara Kitchenham,et al.  The MERMAID Approach to software cost estimation , 1990 .

[4]  Stephen G. MacDonell,et al.  What accuracy statistics really measure , 2001, IEE Proc. Softw..

[5]  Emilia Mendes,et al.  Towards the prediction of development effort for hypermedia applications , 2001, Hypertext.

[6]  Lawrence H. Putnam,et al.  A General Empirical Solution to the Macro Software Sizing and Estimating Problem , 1978, IEEE Transactions on Software Engineering.

[7]  Seishi Okamoto,et al.  An Average-Case Analysis of k-Nearest Neighbor Classifier , 1995, ICCBR.

[8]  Emilia Mendes,et al.  Measurement and Effort Prediction for Web Applications , 2001, Web Engineering.

[9]  Douglas Fisher,et al.  Machine Learning Approaches to Estimating Software Development Effort , 1995, IEEE Trans. Software Eng..

[10]  San Murugesan,et al.  Second ICSE workshop on web engineering , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[11]  Emilia Mendes,et al.  Web Metrics-Estimating Design and Authoring Effort , 2001, IEEE Multim..

[12]  BryantA.,et al.  B. W. Boehm software engineering economics , 1983 .

[13]  Stephen G. MacDonell,et al.  Applications of fuzzy logic to software metric models for development effort estimation , 1997, 1997 Annual Meeting of the North American Fuzzy Information Processing Society - NAFIPS (Cat. No.97TH8297).

[14]  Adam A. Porter,et al.  Learning from Examples: Generation and Evaluation of Decision Trees for Software Resource Analysis , 1988, IEEE Trans. Software Eng..

[15]  Lionel C. Briand,et al.  A replicated assessment and comparison of common software cost modeling techniques , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[16]  Robert T. Hughes,et al.  An empirical investigation into the estimation of software development effort , 1997 .

[17]  Tom DeMarco,et al.  Controlling Software Projects: Management, Measurement, and Estimates , 1986 .

[18]  Emilia Mendes,et al.  Measurement, prediction and risk analysis for Web applications , 2001, Proceedings Seventh International Software Metrics Symposium.

[19]  Xiangzhu Gao,et al.  Assessing Software Cost Estimation Models: criteria for accuracy, consistency and regression , 1997, Australas. J. Inf. Syst..

[20]  D. Ross Jeffery,et al.  Using public domain metrics to estimate software development effort , 2001, Proceedings Seventh International Software Metrics Symposium.

[21]  D. Ross Jeffery,et al.  A comparative study of two software development cost modeling techniques using multi-organizational and company-specific data , 2000, Inf. Softw. Technol..

[22]  Susana Pajares Tosca.  The lyrical quality of links , 1999, HYPERTEXT '99.

[23]  Larry Schroeder,et al.  Understanding Regression Analysis: An Introductory Guide , 2016 .

[24]  Adrian J. C. Cowderoy,et al.  A metrics framework for multimedia creation , 1998, Proceedings Fifth International Software Metrics Symposium. Metrics (Cat. No.98TB100262).

[25]  Chris F. Kemerer,et al.  An empirical validation of software cost estimation models , 1987, CACM.

[26]  Watts S. Humphrey,et al.  A discipline for software engineering , 2012, Series in software engineering.

[27]  Stephen G. MacDonell,et al.  Factors systematically associated with errors in subjective estimates of software development effort: the stability of expert judgment , 1999, Proceedings Sixth International Software Metrics Symposium (Cat. No.PR00403).

[28]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[29]  Barbara A. Kitchenham,et al.  Effort estimation using analogy , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[30]  Sylvie Ranwez,et al.  Formalization to improve lifelong learning , 2000 .

[31]  David L. Sjoquist,et al.  Understanding Regression Analysis , 1986 .

[32]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[33]  Adrian Cowderoy.  Measures of size and complexity for web-site content , 2000 .

[34]  Lionel C. Briand,et al.  An assessment and comparison of common software cost estimation modeling techniques , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[35]  Jean-Marc Desharnais,et al.  A comparison of software effort estimation techniques: Using function points with neural networks, case-based reasoning and regression models , 1997, J. Syst. Softw..

[36]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[37]  Ioannis Stamelos,et al.  A Simulation Tool for Efficient Analogy Based Cost Estimation , 2000, Empirical Software Engineering.

[38]  S. Gentil,et al.  Expected benefits of web-based learning for engineering education: Examples in control engineering , 2001 .

[39]  Michelle Cartwright,et al.  Issues on the Effective Use of CBR Technology for Software Project Prediction , 2001, ICCBR.

[40]  Roy T. Fielding,et al.  Principled design of the modern Web architecture , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[41]  Martin Shepperd,et al.  Using Simulation to Evaluate Prediction Techniques , 2001 .

[42]  Philip M. Johnson,et al.  A Critical Analysis of PSP Data Quality: Results from a Case Study , 1999, Empirical Software Engineering.

[43]  Satish Kumar,et al.  Fuzzy systems and neural networks in software engineering project management , 1994, Applied Intelligence.

[44]  Stephen G. MacDonell,et al.  A comparison of techniques for developing predictive models of software metrics , 1997, Inf. Softw. Technol..

[45]  Martin J. Shepperd,et al.  Estimating Software Project Effort Using Analogies , 1997, IEEE Trans. Software Eng..

[46]  Emilia Mendes,et al.  A Comparison of Length , Complexity and Functionality as Size Measures for Predicting Web Design and Authoring Effort , 2001 .

[47]  Ingunn Myrtveit,et al.  A Controlled Experiment to Assess the Benefits of Estimating with Analogy and Regression Models , 1999, IEEE Trans. Software Eng..