Web hypermedia cost estimation: further assessment and comparison off cost estimation modelling techniques

Abstract Research into Web cost estimation is relatively new, where few studies have compared cost estimation modelling techniques for Web development, with an emphasis placed on techniques such as Case-based Reasoning (CBR), linear and stepwise regression. Although in a large subset of these studies CBR has given the best predictions, results were based on a simple type of CBR, where no adaptation rules were used to adjust the estimated effort obtained. In addition, when comparing the prediction accuracy of estimation models, analysis has been limited to a maximum of three training/validation sets, which according to recent studies, may lead to untrustworthy results. Since CBR is potentially easier to understand and apply (two important factors to the successful adoption of estimation methods within Web development companies), it should be examined further. This paper has therefore two objectives: i) to further investigate the use of CBR for Web development effort prediction by comparing effort prediction accuracy of several CBR techniques; ii) to compare the effort prediction accuracy of the best CBR technique against stepwise and multiple linear regression, using twenty combinations of training/validation sets. Various measures of effort prediction accuracy were applied. One dataset was used in the estimation process. Stepwise and multiple linear regression showed the best prediction accuracy for the dataset employed.

[1]  Ioannis Stamelos,et al.  A Simulation Tool for Efficient Analogy Based Cost Estimation , 2000, Empirical Software Engineering.

[2]  Barbara A. Kitchenham,et al.  Effort estimation using analogy , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[3]  Tomás Isakowitz,et al.  RMM: a methodology for structured hypermedia design , 1995, CACM.

[4]  D. Ross Jeffery,et al.  An Empirical Study of Analogy-based Software Effort Estimation , 1999, Empirical Software Engineering.

[5]  Michelle Cartwright,et al.  Issues on the Effective Use of CBR Technology for Software Project Prediction , 2001, ICCBR.

[6]  Lionel C. Briand,et al.  An assessment and comparison of common software cost estimation modeling techniques , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[7]  Emilia Mendes,et al.  A Comparison of Length , Complexity and Functionality as Size Measures for Predicting Web Design and Authoring Effort , 2001 .

[8]  Ingunn Myrtveit,et al.  A Controlled Experiment to Assess the Benefits of Estimating with Analogy and Regression Models , 1999, IEEE Trans. Software Eng..

[9]  Franca Garzotto,et al.  Towards a software engineering approach to Web site development , 1998, Proceedings Ninth International Workshop on Software Specification and Design.

[10]  Martin J. Shepperd,et al.  Making inferences with small numbers of training sets , 2002, IEE Proc. Softw..

[11]  Emilia Mendes,et al.  A comparison of development effort estimation techniques for Web hypermedia applications , 2002, Proceedings Eighth IEEE Symposium on Software Metrics.

[12]  Stefan Schulz CBR-Works A State-of-the-Art Shell for Case-Based Application Building , 1999 .

[13]  Barbara Kitchenham,et al.  The MERMAID Approach to software cost estimation , 1990 .

[14]  Barbara A. Kitchenham,et al.  An investigation of analysis techniques for software datasets , 1999, Proceedings Sixth International Software Metrics Symposium (Cat. No.PR00403).

[15]  Adrian Cowderoy Measures of size and complexity for web-site content , 2000 .

[16]  José Javier Dolado,et al.  On the problem of the software cost function , 2001, Inf. Softw. Technol..

[17]  Sotiris P. Christodoulou,et al.  Web Engineering: The Developers' View and a Practitioner's Approach , 2001, Web Engineering.

[18]  Martin Shepperd,et al.  Using Simulation to Evaluate Prediction Techniques , 2001 .

[19]  Martin Shepperd,et al.  Experiences Using Case-Based Reasoning to Predict Software Project Effort , 2000 .

[20]  Emilia Mendes,et al.  Measurement and Effort Prediction for Web Applications , 2001, Web Engineering.

[21]  Adrian J. C. Cowderoy,et al.  A metrics framework for multimedia creation , 1998, Proceedings Fifth International Software Metrics Symposium. Metrics (Cat. No.98TB100262).

[22]  Joonhee Yoo,et al.  A systematic approach to designing a WWW application , 1995, CACM.

[23]  Chris F. Kemerer,et al.  An empirical validation of software cost estimation models , 1987, CACM.

[24]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[25]  Franca Garzotto,et al.  HDM—a model-based approach to hypertext application design , 1993, TOIS.

[26]  Ian D. Watson,et al.  Applying case-based reasoning - techniques for the enterprise systems , 1997 .

[27]  Seishi Okamoto,et al.  An Average-Case Analysis of k-Nearest Neighbor Classifier , 1995, ICCBR.

[28]  Gustavo Rossi,et al.  From Domain Models to Hypermedia Applications: an Object-Oriented Approach , 1994 .

[29]  Emilia Mendes,et al.  Web Metrics-Estimating Design and Authoring Effort , 2001, IEEE Multim..

[30]  Emilia Mendes,et al.  A comparison of case-based reasoning approaches , 2002, WWW '02.

[31]  David L. Sjoquist,et al.  Understanding Regression Analysis , 1986 .

[32]  D. Ross Jeffery,et al.  Using public domain metrics to estimate software development effort , 2001, Proceedings Seventh International Software Metrics Symposium.

[33]  Stephen G. MacDonell,et al.  What accuracy statistics really measure , 2001, IEE Proc. Softw..

[34]  Donald J. Reifer,et al.  Web Development: Estimating Quick-to-Market Software , 2000, IEEE Softw..