Definition and evaluation of a COSMIC measurement procedure for sizing Web applications in a model-driven development environment

Abstract Context. Model-driven development approaches facilitate the production of Web applications. Among them, the Object-Oriented Hypermedia method (OO-H) has been successfully used for the development of industrial Web applications. Similarly to other development approaches, it is important also in this context to put measures in place to support project managers in resource allocation, cost and schedule control, and productivity monitoring. Objective. This motivated us to define a measurement procedure, named OO-HCFP, specifically conceived for OO-H Web applications based on COSMIC, a second-generation functional size measurement method. Method. We present mapping and measurement rules devised to automatically derive size measures from OO-H models. We also carry out an empirical study to evaluate whether our proposed measurement procedure, OO-HCFP, is useful for estimating the effort needed to realise industrial Web applications developed with OO-H. Results. The estimates obtained by using OO-HCFP are more accurate than those obtained by using other measurement approaches based on Function Points and design measures. Conclusions. The proposed approach can be profitably exploited to size Web applications developed with OO-H. Based on our experience, we also provide some guidelines to support the formulation of COSMIC measurement procedures for other model-driven approaches.

[1]  Filomena Ferrucci,et al.  Approximate COSMIC Size to Early Estimate Web Application Development Effort , 2013, 2013 39th Euromicro Conference on Software Engineering and Advanced Applications.

[2]  Filomena Ferrucci,et al.  Which COSMIC Base Functional Components are Significant in Estimating Web Application Development? - A Case Study , 2010 .

[3]  Silvia Mara Abrahão,et al.  Validating a size measure for effort estimation in model-driven Web development , 2010, Inf. Sci..

[4]  Boris Milasinovic,et al.  Early effort estimation in web application development , 2015, J. Syst. Softw..

[5]  Genny Tortora,et al.  A COSMIC-FFP Approach to Predict Web Application Development Effort , 2006, J. Web Eng..

[6]  Filomena Ferrucci,et al.  Using Web Objects for Development Effort Estimation of Web Applications: A Replicated Study , 2011, PROFES.

[7]  Oscar Pastor,et al.  Measuring the functional size of web applications , 2003, Int. J. Web Eng. Technol..

[8]  Silvia Mara Abrahão,et al.  Functional versus design measures for model-driven Web applications: A case study in the context of Web effort estimation , 2012, 2012 3rd International Workshop on Emerging Trends in Software Metrics (WETSoM).

[9]  Barbara Tversky,et al.  Visualizing Thought , 2011, Top. Cogn. Sci..

[10]  Austen Rainer,et al.  Case Study Research in Software Engineering - Guidelines and Examples , 2012 .

[11]  Mark Rouncefield,et al.  Model-driven engineering practices in industry , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[12]  Mark Rouncefield,et al.  Model-driven engineering practices in industry: Social, organizational and managerial factors that lead to success or failure , 2014, Sci. Comput. Program..

[13]  Oscar Pastor,et al.  Towards an accurate functional size measurement procedure for conceptual models in an MDA environment , 2010, Data Knowl. Eng..

[14]  Filomena Ferrucci,et al.  Genetic Programming for Effort Estimation: An Analysis of the Impact of Different Fitness Functions , 2010, 2nd International Symposium on Search Based Software Engineering.

[15]  Mark Harman,et al.  Multi-objective Software Effort Estimation , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[16]  Emilia Mendes,et al.  Cross- vs. within-company cost estimation studies revisited: an extended systematic review , 2014, EASE '14.

[17]  Michele Marchesi,et al.  Web framework points: an effort estimation methodology for Web application development using a content management framework , 2013, J. Softw. Evol. Process..

[19]  Roel Wieringa,et al.  Design Science Methodology for Information Systems and Software Engineering , 2014, Springer Berlin Heidelberg.

[20]  Filomena Ferrucci,et al.  From Function Points to COSMIC - A Transfer Learning Approach for Effort Estimation , 2015, PROFES.

[21]  Filomena Ferrucci,et al.  Web Effort Estimation: Function Point Analysis vs. COSMIC , 2016, Inf. Softw. Technol..

[22]  Emilia Mendes,et al.  Using tabu search to configure support vector regression for effort estimation , 2013, Empirical Software Engineering.

[23]  Oscar Pastor,et al.  Automating the Measurement of Functional Size of Conceptual Models in an MDA Environment , 2008, PROFES.

[24]  Roel Wieringa,et al.  Six strategies for generalizing software engineering theories , 2015, Sci. Comput. Program..

[25]  Silvia Mara Abrahão,et al.  A model-driven measurement procedure for sizing web applications: design, automation and validation , 2007, MODELS'07.

[26]  Alain Abran,et al.  Design of a Functional Size Measurement Procedure for Real-Time Embedded Software Requirements Expressed using the Simulink Model , 2011, 2011 Joint Conference of the 21st International Workshop on Software Measurement and the 6th International Conference on Software Process and Product Measurement.

[27]  Filomena Ferrucci,et al.  Using Evolutionary Based Approaches to Estimate Software Development Effort , 2010 .

[28]  Piero Fraternali,et al.  Automating function point analysis with model driven development , 2006, CASCON.

[29]  Çigdem Gencel,et al.  Impact of Base Functional Component Types on Software Functional Size Based Effort Estimation , 2008, PROFES.

[30]  Filomena Ferrucci,et al.  Investigating Functional and Code Size Measures for Mobile Applications , 2015, 2015 41st Euromicro Conference on Software Engineering and Advanced Applications.

[31]  Federica Sarro.  Search-based approaches for software development effort estimation , 2011 .

[32]  B. Kitchenham,et al.  Case Studies for Method and Tool Evaluation , 1995, IEEE Softw..

[33]  Isabella Wieczorek,et al.  Resource Estimation in Software Engineering , 2002 .

[34]  S. Abrahão,et al.  A COSMIC measurement procedure for sizing web applications developed using the OO-H method , 2010 .

[35]  Marco Brambilla,et al.  Designing Web Applications with Webml and Webratio , 2008, Web Engineering.

[36]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2012, Springer Berlin Heidelberg.

[37]  Burak Turhan,et al.  A Comparison of Cross-Versus Single-Company Effort Prediction Models for Web Projects , 2014, 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications.

[38]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[39]  Douglas Fisher,et al.  Machine Learning Approaches to Estimating Software Development Effort , 1995, IEEE Trans. Software Eng..

[40]  Vinay Kulkarni,et al.  Cost estimation for model-driven engineering , 2012, MODELS'12.

[41]  Barbara A. Kitchenham,et al.  A Procedure for Analyzing Unbalanced Datasets , 1998, IEEE Trans. Software Eng..

[42]  Tayana Conte,et al.  Realising web effort estimation: a qualitative investigation , 2013, EASE '13.

[43]  Oscar Pastor,et al.  Faculteit Economie En Bedrijfskunde Hoveniersberg 24 B-9000 Gent Functional Size Measurement Method for Object-oriented Conceptual Schemas: Design and Evaluation Issues a Functional Size Measurement Method for Object-oriented Conceptual Schemas: Design and Evaluation Issues , 2022 .

[44]  Tore Dybå,et al.  A systematic review of effect size in software engineering experiments , 2007, Inf. Softw. Technol..

[45]  Lionel C. Briand,et al.  Modeling Development Effort in Object-Oriented Systems Using Design Properties , 2001, IEEE Trans. Software Eng..

[46]  Harold S. van Heeringen,et al.  Measure the Functional Size of a Mobile App: Using the COSMIC Functional Size Measurement Method , 2014, 2014 Joint Conference of the International Workshop on Software Measurement and the International Conference on Software Process and Product Measurement.

[47]  Mark Rouncefield,et al.  Empirical assessment of MDE in industry , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[48]  Nelly Condori-Fernández,et al.  Verifying the Construction of a Software Model from a Requirements Model , 2006, WER.

[49]  Cristina Cachero,et al.  Conceptual Modeling of Device-Independent Web Applications , 2001, JISBD.

[50]  Emilia Mendes,et al.  Investigating Tabu Search for Web Effort Estimation , 2010, 2010 36th EUROMICRO Conference on Software Engineering and Advanced Applications.

[51]  Emilia Mendes,et al.  A systematic review of web resource estimation , 2012, PROMISE '12.

[52]  Emilia Mendes,et al.  Further comparison of cross-company and within-company effort estimation models for Web applications , 2004 .

[53]  Patrick van Bommel.  Information modeling for internet applications , 2003 .

[54]  C. Symons,et al.  The COSMIC Functional Size Measurement Method : Measurement Manual : The COSMIC Implementation Guide for ISO/IEC 19761:2011 , 2015 .

[55]  Shari Lawrence Pfleeger,et al.  Preliminary Guidelines for Empirical Research in Software Engineering , 2002, IEEE Trans. Software Eng..

[56]  Oscar Pastor,et al.  Measurement of Functional Size in Conceptual Models: A Survey of Measurement Procedures Based on COSMIC , 2008, IWSM/Metrikon/Mensura.

[57]  Dirk Meyerhoff,et al.  Software Quality and Software Testing in Internet Times , 2002, Springer Berlin Heidelberg.

[58]  Silvia Mara Abrahão,et al.  A family of experiments to evaluate a functional size measurement procedure for Web applications , 2009, J. Syst. Softw..

[59]  Luciano Baresi,et al.  Three empirical studies on estimating the design effort of Web applications , 2007, TSEM.

[60]  Michele Marchesi,et al.  Size estimation of Web applications through Web CMF Object , 2012, 2012 3rd International Workshop on Emerging Trends in Software Metrics (WETSoM).

[61]  Emilia Mendes,et al.  Investigating Web size metrics for early Web cost estimation , 2005, J. Syst. Softw..

[62]  Sandro Morasca,et al.  On the use of weighted sums in the definition of measures , 2010, WETSoM '10.

[63]  Filomena Ferrucci,et al.  Estimating Web Application Development Effort Using Web-COBRA and COSMIC: An Empirical Study , 2009, 2009 35th Euromicro Conference on Software Engineering and Advanced Applications.

[64]  Filomena Ferrucci,et al.  Using Tabu Search to Estimate Software Development Effort , 2009, IWSM/Mensura.

[65]  Robert L. Glass.  Software Engineering: Facts and Fallacies , 2002 .

[66]  D. Ross Jeffery,et al.  Using Web objects for estimating software development effort for Web applications , 2003, Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717).

[67]  Onur Demirörs,et al.  Functional size measurement revisited , 2008, TSEM.

[68]  Luciano Baresi,et al.  From Web Sites to Web Applications: New Issues for Conceptual Modeling , 2000, ER.

[69]  Suad Kasapovic,et al.  Web application functional size estimation based on COSMIC method and UWE approach , 2013, 2013 36th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[70]  Emilia Mendes,et al.  How to Make Best Use of Cross-Company Data for Web Effort Estimation? , 2015, 2015 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).

[71]  Filomena Ferrucci,et al.  How Multi-Objective Genetic Programming Is Effective for Software Development Effort Estimation? , 2011, SSBSE.

[72]  Tayana Conte,et al.  Is there a place for qualitative studies when identifying effort predictors?: a case in web effort estimation , 2014, EASE '14.

[73]  Tim Menzies,et al.  Transfer learning in effort estimation , 2015, Empirical Software Engineering.

[74]  Ó. Pastor,et al.  On Conceptual Modeling of Device-Independent Web Applications : Towards a Web Engineering Approach , 2000 .

[75]  Marco Brambilla,et al.  Dissecting Design Effort and Drawing Effort in UML Modeling , 2017, 2017 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA).

[76]  Antonio Vallecillo,et al.  Towards interoperable Web engineering methods , 2008, J. Assoc. Inf. Sci. Technol..

[77]  Stephen G. MacDonell,et al.  Evaluating prediction systems in software project estimation , 2012, Inf. Softw. Technol..

[78]  Filomena Ferrucci,et al.  A Case Study Using Web Objects and COSMIC for Effort Estimation of Web Applications , 2008, 2008 34th Euromicro Conference Software Engineering and Advanced Applications.

[79]  Filomena Ferrucci,et al.  Investigating Functional and Code Size Measures for Mobile Applications , 2015, EUROMICRO-SEAA.

[80]  Michele Marchesi,et al.  Effort Estimation of Web Applications through Web CMF Objects , 2012, 2012 Joint Conference of the 22nd International Workshop on Software Measurement and the 2012 Seventh International Conference on Software Process and Product Measurement.

[81]  Mark Harman,et al.  Search-Based Software Project Management , 2014, Software Project Management in a Changing World.

[82]  Emilia Mendes,et al.  How effective is Tabu search to configure support vector regression for effort estimation? , 2010, PROMISE '10.

[83]  Stefano Ceri,et al.  Web Modeling Language (WebML): a modeling language for designing Web sites , 2000, Comput. Networks.

[84]  Mark Harman,et al.  Exact Mean Absolute Error of Baseline Predictor, MARP0 , 2016, Inf. Softw. Technol..

[85]  Ingunn Myrtveit,et al.  Validity and reliability of evaluation procedures in comparative studies of effort prediction models , 2011, Empirical Software Engineering.

[86]  Stephen G. MacDonell,et al.  What accuracy statistics really measure , 2001, IEE Proc. Softw..

[87]  Emilia Mendes,et al.  Web effort estimation: the value of cross-company data set compared to single-company data set , 2012, PROMISE '12.

[88]  J. Royston.  An Extension of Shapiro and Wilk's W Test for Normality to Large Samples , 1982 .

[89]  Filomena Ferrucci,et al.  Single and Multi Objective Genetic Programming for software development effort estimation , 2012, SAC '12.

[90]  Tim Menzies,et al.  On the Value of Ensemble Effort Estimation , 2012, IEEE Transactions on Software Engineering.

[91]  Emilia Mendes.  Using knowledge elicitation to improve Web effort estimation: Lessons from six industrial case studies , 2012, 2012 34th International Conference on Software Engineering (ICSE).