Web Effort Estimation: Function Point Analysis vs. COSMIC

Context: software development effort estimation is a crucial management task that critically depends on the adopted size measure. Several Functional Size Measurement (FSM) methods have been proposed. COSMIC is considered a 2nd generation FSM method, to differentiate it from Function Point Analysis (FPA) and its variants, considered as 1st generation ones. In the context of Web applications, few investigations have been performed to compare the effectiveness of the two generations. Software companies could benefit from this analysis to evaluate if it is worth to migrate from a 1st generation method to a 2nd one.Objective: the main goal of the paper is to empirically investigate if COSMIC is more effective than FPA for Web effort estimation. Since software companies using FPA cannot build an estimation model based on COSMIC as long as they do not have enough COSMIC data, the second goal of the paper is to investigate if conversion equations can be exploited to support the migration from FPA to COSMIC.Method: two empirical studies have been carried out by employing an industrial data set. The first one compared the effort prediction accuracy obtained with Function Points (FPs) and COSMIC, using two estimation techniques (Simple Linear Regression and Case-Based Reasoning). The second study assessed the effectiveness of a two-step strategy that first exploits a conversion equation to transform historical FPs data into COSMIC, and then builds a new prediction model based on those estimated COSMIC sizes.Results: the first study revealed that, on our data set, COSMIC was significantly more accurate than FPs in estimating the development effort. The second study revealed that the effectiveness of the analyzed two-step process critically depends on the employed conversion equation.Conclusion: for Web effort estimation COSMIC can be significantly more effective than FPA. Nevertheless, additional research must be conducted to identify suitable conversion equations so that the two-step strategy can be effectively employed for a smooth migration from FPA to COSMIC.

[1]  Onur Demirörs,et al.  Functional size measurement revisited , 2008, TSEM.

[2]  Cem Kaner,et al.  Software Engineering Metrics: What Do They Measure and How Do We Know? , 2004 .

[3]  Emilia Mendes,et al.  Cross-company vs. single-company web effort models using the Tukutuku database: An extended study , 2008, J. Syst. Softw..

[4]  Ingunn Myrtveit,et al.  Validity and reliability of evaluation procedures in comparative studies of effort prediction models , 2011, Empirical Software Engineering.

[5]  C. Gravino,et al.  Estimating Web Application Development Effort Using Cosmic-Ffp Method , 2009 .

[6]  Elizabeth A. Peck,et al.  Introduction to Linear Regression Analysis , 2001 .

[7]  Sandro Morasca,et al.  Convertibility of Function Points into COSMIC Function Points: A study using Piecewise Linear Regression , 2011, Inf. Softw. Technol..

[8]  Tim Menzies,et al.  oftware effort models should be assessed via leave-one-out validation , 2013 .

[9]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[10]  Tim Menzies,et al.  On the Value of Ensemble Effort Estimation , 2012, IEEE Transactions on Software Engineering.

[11]  Mark Harman,et al.  Search-Based Software Project Management , 2014, Software Project Management in a Changing World.

[12]  S. Abrahão,et al.  A COSMIC measurement procedure for sizing web applications developed using the OO-H method , 2010 .

[13]  Jeffrey C. Carver,et al.  Issues in using students in empirical studies in software engineering education , 2003, Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717).

[14]  D. Ross Jeffery,et al.  Cost estimation for web applications , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[15]  Emilia Mendes,et al.  Comparison of Web size measures for predicting Web design and authoring effort , 2002, IEE Proc. Softw..

[16]  Filomena Ferrucci,et al.  Approximate COSMIC Size to Early Estimate Web Application Development Effort , 2013, 2013 39th Euromicro Conference on Software Engineering and Advanced Applications.

[17]  Emilia Mendes,et al.  A Comparative Study of Cost Estimation Models for Web Hypermedia Applications , 2003, Empirical Software Engineering.

[18]  Stephen G. MacDonell,et al.  Evaluating prediction systems in software project estimation , 2012, Inf. Softw. Technol..

[19]  Filomena Ferrucci,et al.  Which COSMIC Base Functional Components are Significant in Estimating Web Application Development? - A Case Study , 2010 .

[20]  Filomena Ferrucci,et al.  A Case Study Using Web Objects and COSMIC for Effort Estimation of Web Applications , 2008, 2008 34th Euromicro Conference Software Engineering and Advanced Applications.

[21]  Luciano Baresi,et al.  Three empirical studies on estimating the design effort of Web applications , 2007, TSEM.

[22]  Filomena Ferrucci,et al.  Using Web Objects for Development Effort Estimation of Web Applications: A Replicated Study , 2011, PROFES.

[23]  Antonio Vallecillo,et al.  A UML 2.0 profile for WebML modeling , 2006, ICWE '06.

[24]  Emilia Mendes,et al.  How to Make Best Use of Cross-Company Data for Web Effort Estimation? , 2015, 2015 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).

[25]  Oscar Pastor,et al.  An Object-Oriented Approach to Automate Web Applications Development , 2001, EC-Web.

[26]  Stephen G. MacDonell,et al.  What accuracy statistics really measure , 2001, IEE Proc. Softw..

[27]  Donald J. Reifer.  Estimating Web Development Costs: There Are Differences , 2002 .

[28]  J. Royston.  An Extension of Shapiro and Wilk's W Test for Normality to Large Samples , 1982 .

[29]  Adam Trendowicz,et al.  Software Project Effort Estimation: Foundations and Best Practice Guidelines for Success , 2014 .

[30]  Emilia Mendes,et al.  Effort estimation: how valuable is it for a web company to use a cross-company data set, compared to using its own single-company data set? , 2007, WWW '07.

[31]  John E. Gaffney,et al.  Software Function, Source Lines of Code, and Development Effort Prediction: A Software Science Validation , 1983, IEEE Transactions on Software Engineering.

[32]  Filomena Ferrucci,et al.  Estimating Web Application Development Effort Using Web-COBRA and COSMIC: An Empirical Study , 2009, 2009 35th Euromicro Conference on Software Engineering and Advanced Applications.

[33]  Emilia Mendes,et al.  Cross- vs. within-company cost estimation studies revisited: an extended systematic review , 2014, EASE '14.

[34]  Emilia Mendes,et al.  Investigating Web size metrics for early Web cost estimation , 2005, J. Syst. Softw..

[35]  Karen T. Lum,et al.  Selecting Best Practices for Effort Estimation , 2006, IEEE Transactions on Software Engineering.

[36]  Genny Tortora,et al.  A COSMIC-FFP Approach to Predict Web Application Development Effort , 2006, J. Web Eng..

[37]  D. Ross Jeffery,et al.  Using Web objects for estimating software development effort for Web applications , 2003, Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717).

[38]  Oscar Pastor,et al.  Measuring the functional size of web applications , 2003, Int. J. Web Eng. Technol..

[39]  Emilia Mendes,et al.  Further comparison of cross-company and within-company effort estimation models for Web applications , 2004 .

[40]  Emilia Mendes,et al.  Investigating the use of Support Vector Regression for web effort estimation , 2011, Empirical Software Engineering.

[41]  Emilia Mendes,et al.  Using tabu search to configure support vector regression for effort estimation , 2013, Empirical Software Engineering.

[42]  T. Breurch,et al.  A simple test for heteroscedasticity and random coefficient variation (econometrica vol 47 , 1979 .

[43]  Barbara A. Kitchenham.  The Problem with Function Points , 1997, IEEE Software.

[44]  Silvia Mara Abrahão,et al.  Functional versus design measures for model-driven Web applications: A case study in the context of Web effort estimation , 2012, 2012 3rd International Workshop on Emerging Trends in Software Metrics (WETSoM).

[45]  Thomas Fetcke,et al.  The Warehouse Software Portfolio A Case Study in Functional Size Measurement , 1999 .

[46]  Lefteris Angelis,et al.  Using Ensembles for Web Effort Estimation , 2013, 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement.

[47]  Piero Fraternali,et al.  Automating function point analysis with model driven development , 2006, CASCON.

[48]  Çigdem Gencel,et al.  Exploring the Convertibility between IFPUG and COSMIC Function Points: Preliminary Findings , 2012, 2012 Joint Conference of the 22nd International Workshop on Software Measurement and the 2012 Seventh International Conference on Software Process and Product Measurement.

[49]  Geng Liu,et al.  Analytical Convertibility of Functional Size Measures: A Tool-based Approach , 2012, 2012 Joint Conference of the 22nd International Workshop on Software Measurement and the 2012 Seventh International Conference on Software Process and Product Measurement.

[50]  Abdul Azim Abdul Ghani,et al.  An exploratory study on the accuracy of FPA to COSMIC measurement method conversion types , 2012, Inf. Softw. Technol..

[51]  Martin J. Shepperd,et al.  Estimating Software Project Effort Using Analogies , 1997, IEEE Trans. Software Eng..

[52]  Alain Abran,et al.  Measurement convertibility : from function points to COSMIC-FFP , 2005 .

[53]  Ingunn Myrtveit,et al.  A Controlled Experiment to Assess the Benefits of Estimating with Analogy and Regression Models , 1999, IEEE Trans. Software Eng..

[54]  Guilherme Horta Travassos,et al.  Cross versus Within-Company Cost Estimation Studies: A Systematic Review , 2007, IEEE Transactions on Software Engineering.

[55]  Silvia Mara Abrahão,et al.  A model-driven measurement procedure for sizing web applications: design, automation and validation , 2007, MODELS'07.

[56]  Alain Abran,et al.  Function points: A study of their measurement processes and scale transformations , 1994, J. Syst. Softw..

[57]  Frank Vogelezang,et al.  Applicability of COSMIC in an administrative environment: Experiences of an early adopter , 2003 .

[58]  Alain Abran,et al.  Convertibility of function points to COSMIC-FPP: identification and analysis fo functional outliers , 2006 .

[59]  Horst Zuse,et al.  A Framework of Software Measurement , 1998 .

[60]  Luigi Lavazza.  An evaluation of the statistical convertibility of Function Points into COSMIC Function Points , 2013, Empirical Software Engineering.

[61]  N. Mantel.  The detection of disease clustering and a generalized regression approach. , 1967, Cancer research.

[62]  Lionel C. Briand,et al.  Modeling Development Effort in Object-Oriented Systems Using Design Properties , 2001, IEEE Trans. Software Eng..

[63]  Emilia Mendes,et al.  Comparing Size Measures for Predicting Web Application Development Effort: A Case Study , 2007, ESEM 2007.

[64]  Onur Demirörs,et al.  An experimental study on the conversion between IFPUG and COSMIC functional size measurement units , 2010, Inf. Softw. Technol..

[65]  Alain Abran.  Convertibility across Measurement Methods , 2010 .

[66]  Emilia Mendes,et al.  Bayesian Network Models for Web Effort Prediction: A Comparative Study , 2008, IEEE Transactions on Software Engineering.

[67]  Bruce McMillin,et al.  Software engineering: What is it? , 2018, 2018 IEEE Aerospace Conference.

[68]  Frank Bomarius,et al.  COBRA: a hybrid method for software cost estimation, benchmarking, and risk assessment , 1998, Proceedings of the 20th International Conference on Software Engineering.

[69]  Tore Dybå,et al.  A systematic review of effect size in software engineering experiments , 2007, Inf. Softw. Technol..