Do adaptation rules improve web cost estimation?

Analogy-based estimation has, over the last 15 years, and particularly over the last 7 years, emerged as a promising approach with comparable accuracy to, or better than, algorithmic methods in some studies. In addition, it is potentially easier to understand and apply; these two important factors can contribute to the successful adoption of estimation methods within Web development companies. We believe therefore, analogy-based estimation should be examined further.This paper compares several methods of analogy-based effort estimation. In particular, it investigates the use of adaptation rules as a contributing factor to better estimation accuracy. Two datasets are used in the analysis; results show that the best predictions are obtained for the dataset that first, presents a continuous "cost" function, translated as a strong linear relationship between size and effort, and second, is more "unspoiled" in terms of outliers and collinearity. Only one of the two types of adaptation rules employed generated good predictions.

[1]  Barbara A. Kitchenham,et al.  Effort estimation using analogy , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[2]  Emilia Mendes,et al.  The application of case-based reasoning to early Web project cost estimation , 2002, Proceedings 26th Annual International Computer Software and Applications.

[3]  Martin Shepperd,et al.  Case and Feature Subset Selection in Case-Based Software Project Effort Prediction , 2003 .

[4]  F KemererChris.  An empirical validation of software cost estimation models , 1987 .

[5]  Barbara A. Kitchenham,et al.  A Further Empirical Investigation of the Relationship Between MRE and Project Size , 2003, Empirical Software Engineering.

[6]  Barbara A. Kitchenham,et al.  An investigation of analysis techniques for software datasets , 1999, Proceedings Sixth International Software Metrics Symposium (Cat. No.PR00403).

[7]  Adrian Cowderoy.  Measures of size and complexity for web-site content , 2000 .

[8]  D. Ross Jeffery,et al.  Calibrating estimation tools for software development , 1990, Softw. Eng. J..

[9]  Satish Kumar,et al.  Fuzzy systems and neural networks in software engineering project management , 1994, Applied Intelligence.

[10]  Martin J. Shepperd,et al.  Search Heuristics, Case-based Reasoning And Software Project Effort Prediction , 2002, GECCO.

[11]  Ben Shneiderman,et al.  Structural analysis of hypertexts: identifying hierarchies and useful metrics , 1992, TOIS.

[12]  Martin Shepperd,et al.  Experiences Using Case-Based Reasoning to Predict Software Project Effort , 2000 .

[13]  Lionel C. Briand,et al.  Resource modeling in software engineering , 2002 .

[14]  Emilia Mendes,et al.  Measurement and Effort Prediction for Web Applications , 2001, Web Engineering.

[15]  Adam A. Porter,et al.  Learning from Examples: Generation and Evaluation of Decision Trees for Software Resource Analysis , 1988, IEEE Trans. Software Eng..

[16]  Michael J. Prietula,et al.  Software-effort estimation with a case-based reasoner , 1996, J. Exp. Theor. Artif. Intell..

[17]  Stephen G. MacDonell,et al.  A comparison of model building techniques to develop predictive equations for software metrics , 1997 .

[18]  Seishi Okamoto,et al.  An Average-Case Analysis of k-Nearest Neighbor Classifier , 1995, ICCBR.

[19]  Douglas Fisher,et al.  Machine Learning Approaches to Estimating Software Development Effort , 1995, IEEE Trans. Software Eng..

[20]  Stephen G. MacDonell,et al.  What accuracy statistics really measure , 2001, IEE Proc. Softw..

[21]  Ivar Jacobson,et al.  The Unified Modeling Language User Guide , 1998, J. Database Manag..

[22]  Scott W. Ambler,et al.  Lessons in Agility From Internet-Based Development , 2002, IEEE Softw..

[23]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[24]  Luciano Baresi,et al.  Estimating the design effort of Web applications , 2003, Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717).

[25]  Emilia Mendes,et al.  A comparison of case-based reasoning approaches , 2002, WWW '02.

[26]  Donald J. Reifer.  Ten Deadly Risks in Internet and Intranet Software Development , 2002, IEEE Softw..

[27]  Robert T. Hughes,et al.  An empirical investigation into the estimation of software development effort , 1997 .

[28]  Donald J. Reifer,et al.  Web Development: Estimating Quick-to-Market Software , 2000, IEEE Softw..

[29]  Martin Shepperd,et al.  Using Simulation to Evaluate Prediction Techniques , 2001 .

[30]  D. Ross Jeffery,et al.  A comparative study of two software development cost modeling techniques using multi-organizational and company-specific data , 2000, Inf. Softw. Technol..

[31]  Luciano Baresi,et al.  An empirical study on the design effort of Web applications , 2002, Proceedings of the Third International Conference on Web Information Systems Engineering, 2002. WISE 2002..

[32]  Emilia Mendes,et al.  A Comparative Study of Cost Estimation Models for Web Hypermedia Applications , 2003, Empirical Software Engineering.

[33]  Ioannis Stamelos,et al.  A Simulation Tool for Efficient Analogy Based Cost Estimation , 2000, Empirical Software Engineering.

[34]  S. Gentil,et al.  Expected benefits of web-based learning for engineering education: Examples in control engineering , 2001 .

[35]  Albert L. Lederer,et al.  Information systems software cost estimating: a current assessment , 1993, J. Inf. Technol..

[36]  Stephen G. MacDonell,et al.  Applications of fuzzy logic to software metric models for development effort estimation , 1997, 1997 Annual Meeting of the North American Fuzzy Information Processing Society - NAFIPS (Cat. No.97TH8297).

[37]  Michelle Cartwright,et al.  Issues on the Effective Use of CBR Technology for Software Project Prediction , 2001, ICCBR.

[38]  Sylvie Ranwez,et al.  Formalization to improve lifelong learning , 2000 .

[39]  Roy T. Fielding,et al.  Principled design of the modern Web architecture , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[40]  Barbara A. Kitchenham,et al.  An empirical validation of the relationship between the magnitude of relative error and project size , 2002, Proceedings Eighth IEEE Symposium on Software Metrics.

[41]  Emilia Mendes,et al.  A comparison of development effort estimation techniques for Web hypermedia applications , 2002, Proceedings Eighth IEEE Symposium on Software Metrics.

[42]  Tom DeMarco,et al.  Controlling Software Projects: Management, Measurement, and Estimates , 1986 .

[43]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[44]  Stefan Schulz.  CBR-Works A State-of-the-Art Shell for Case-Based Application Building , 1999 .

[45]  Emilia Mendes,et al.  Measurement, prediction and risk analysis for Web applications , 2001, Proceedings Seventh International Software Metrics Symposium.

[46]  Chris F. Kemerer,et al.  An empirical validation of software cost estimation models , 1987, CACM.

[47]  Watts S. Humphrey,et al.  A discipline for software engineering , 2012, Series in software engineering.

[48]  Stephen G. MacDonell,et al.  Factors systematically associated with errors in subjective estimates of software development effort: the stability of expert judgment , 1999, Proceedings Sixth International Software Metrics Symposium (Cat. No.PR00403).

[49]  Susana Pajares Tosca.  The lyrical quality of links , 1999, HYPERTEXT '99.

[50]  Adrian J. C. Cowderoy,et al.  A metrics framework for multimedia creation , 1998, Proceedings Fifth International Software Metrics Symposium. Metrics (Cat. No.98TB100262).

[51]  Emilia Mendes,et al.  An Empirical Analysis of Linear Adaptation Techniques for Case-Based Prediction , 2003, ICCBR.

[52]  Emilia Mendes,et al.  Investigating Early Web Size Measures for Web Cost Estimation , 2005 .

[53]  Ioannis Stamelos,et al.  Measuring functionality and productivity in Web-based applications: a case study , 1999, Proceedings Sixth International Software Metrics Symposium (Cat. No.PR00403).

[54]  Martin J. Shepperd,et al.  Estimating Software Project Effort Using Analogies , 1997, IEEE Trans. Software Eng..

[55]  Ingunn Myrtveit,et al.  A Controlled Experiment to Assess the Benefits of Estimating with Analogy and Regression Models , 1999, IEEE Trans. Software Eng..

[56]  D. Ross Jeffery,et al.  Cost estimation for web applications , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[57]  R. S. Pressman,et al.  What a tangled Web we weave [Web engineering] , 2000 .

[58]  Emilia Mendes,et al.  Web Metrics-Estimating Design and Authoring Effort , 2001, IEEE Multim..

[59]  D. Ross Jeffery,et al.  Using public domain metrics to estimate software development effort , 2001, Proceedings Seventh International Software Metrics Symposium.

[60]  Martin J. Shepperd,et al.  Comparing Software Prediction Techniques Using Simulation , 2001, IEEE Trans. Software Eng..

[61]  Jean-Marc Desharnais,et al.  Estimating Software Development Effort with Case-Based Reasoning , 1997, ICCBR.

[62]  Ian D. Watson,et al.  Applying case-based reasoning - techniques for the enterprise systems , 1997 .

[63]  Barbara Kitchenham,et al.  The MERMAID Approach to software cost estimation , 1990 .

[64]  D. Ross Jeffery,et al.  An Empirical Study of Analogy-based Software Effort Estimation , 1999, Empirical Software Engineering.

[65]  Lionel C. Briand,et al.  An assessment and comparison of common software cost estimation modeling techniques , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[66]  Jean-Marc Desharnais,et al.  A comparison of software effort estimation techniques: Using function points with neural networks, case-based reasoning and regression models , 1997, J. Syst. Softw..

[67]  강문설.  [서평]「The Unified Modeling Language User Guide」 , 1999 .

[68]  Lionel C. Briand,et al.  A replicated assessment and comparison of common software cost modeling techniques , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.